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■^This  report  conta ins  a method  of  approach  and  theoretical  framework 
which  advances  the  state  of  the  art  in  the  design  of  reliable  multivariable 
control  systems,  with  special  emphasis  on  actuator  failures  and  necessary 
actuator  redundancy  levels. 

The  mathematical  model  consists  of  a linear  time  invariant  discrete 
time  dynamical  system.  Configuration  changes  in  the  system  dynamics,  (such 
as  actuator  failures,  repairs,  introduction  of  a back  up  actuator)  are 
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20.  Abstract 


kyovernod  by  a Markov  chaiii  '■'’it  inclul'”'.  '•rms  I r inn  probabilities  from 
one  conf  i yura  t Ion  state  to  inotber.  I'or  fnrmance  index  is  a standard 

quadratic  cost  functional,  over  in  infinite  time  interval. 

If  the  dynamic  system  contains  either  nrocess  white  noise  and/or 
noisy  measurements  of  the  state,  then  tbe  stochastic  optimal  control 
problem  reduces, In  oenera  I , to  a du.i  I I'rnblem.  and  no  analytical  or 
efficient  aloorithmic  solution  is  nnssible.  TIuis.  the  results  are  obtained 
under  the  assuniotion  ot  full  •;rite  v.iriTiile  measiiremen's , and  in  the 
absence  of  additive  process  wnite  noise. 

Under  the  above  assumptions,  the  optimal  stochastic  control  solution 
can  be  obtained.  The  actual  system  configuration  can  be  deduced  with  an 
one  step  delay.  The  calculation  of  the  optimal  control  law  requires  the 
solution  of  a set  of  hiqhlv  coiiplea  Kiccati-like  matrix  difference  equation.; 
if  these  converge  fas  the  terminal  time  goes  to  infinity)  one  has  a 
rel iabl e design  with  switching  feedback  gains,  and,  if  they  diverge,  the 
design  is  unrel iable  and  the  system  cannot  be  stabilized  unless  more 
reliable  actuators  or  more  redundant  actuators  are  employed.  For  the 
reliable  designs,  the  feedback  system  requires  a switching  gain  solution, 
that  is,  whenever  a system  change  is  detected,  the  feedback  gains  must  be 
reconfigured.  On  the  other  hanu,  the  necessary  reconf i gra t ion  gains  can 
be  precomputed,  from  the  off-line  solutions  of  the  R i cca t i - I i ke  matrix 
difference  equations. 

Through  the  use  oi  Cite  matrix  aiscrote  minimum  principle,  a 
suboptimal  solution  can  also  be  obtaineu.  In  this  approach,  one  wishes 
to  avoid  the  reconfiguration  of  the  feedback  system,  and  one  wishes  to 
know  whether  or  not  it  is  possible  to  stabilize  the  system  with  a constant 
feedback  gain,  which  does  not  change  even  if  the  system  changes.  Once 
more  this  can  be  deduced  from  another  set  of  coupled  Riccati-like  matrix 
difference  equations.  If  they  diverge  as  the  terminal  time  goes  to 
infinity,  then  a constant  gain  Implementation  is  unreliable,  because  it 
cannot  stabilize  the  system.  If,  on  the  other  hand,  there  exists  an 
asymptotic  solution  to  this  set  of  Riccati-like  equations  the  a rel iable 
control  system  without  feedback  reconf i gura t i on  can  be  obtained.  The 
implementation  requires  constant  gain  state  variable  feedback,  and  the 
feedback  gains  can  be  calculated  off-line. 
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ABSTRACT 


This  report  contains  a method  of  approach  and  theoretical 
framework  which  advances  the  state  of  the  art  in  the  design  of 
reliable  multivariable  control  systems,  with  special  emphasis  on 
actuator  failures  and  necessary  actuator  redundancy  levels. 

The  mathematical  model  consists  of  a linear  time  invariant 
discrete  time  dynamical  system.  Configuration  changes  in  the 
system  dynamics,  (such  as  actuator  failures,  repairs,  introduction 
of  a back  up  actuator)  are  governed  by  a Markov  chain  that  inclujies 
transition  probabilities  from  one  configuration  state  to  another. 
The  performance  index  is  a standard  quadratic  cost  functional, 
over  an  infinite  time  interval. 

If  the  dyncunic  system  contains  either  process  white  noise 
and/or  noisy  measurements  of  the  state,  then  the  stochastic 
optimal  control  problem  reduces,  in  general,  to  a dual  problem, 
and  no  analytical  or  efficient  algorithmic  solution  is  possible. 
Thus , the  results  are  obtained  under  the  assumption  of  full  state 
variable  measurements,  and  in  the  absence  of  additive  process 
white  noise. 

Under  the  above  assumptions,  the  optimal  stochastic  control 
solution  can  be  obtained.  The  actual  system  configuration  can 
be  deduced  with  an  one  step  delay.  The  calculation  of  the  optimal 
control  law  requires  the  solution  of  a set  of  highly  coupled 
Riccati-like  matrix  difference  equations;  if  these  converge  (as 
the  terminal  time  goes  to  infinity)  one  has  a reliable  design  witli 
switcliing  feedback  gains,  and,  if  they  diverge,  the  design  is 
unreliable  and  the  system  cannot  be  stabilized  unless  more  reliable 
actuators  or  more  redundant  actuators  are  employed.  For  the 
reliable  designs,  the  feedback  system  requires  a switching  gain 
solution,  that  is,  whenever  a system  change  is  detected,  the  feed- 
back gains  must  be  reconfigured.  On  the  other  hand,  the  necessary 
reconfiguration  gains  can  be  precomputed,  from  the  off-line  solu- 
tions of  the  Riccati-like  matrix  difference  equations. 


I 


4 

Through  the  use  of  the  matrix  discrete  minimum  principle,  a 
suboptimal  solution  can  also  be  obtained.  In  this  approach,  one 
wishes  to  avoid  the  reconfiguration  of  the  feedback  system,  and 
one  wishes  to  know  whether  or  not  it  is  possible  to  stabilize  the 
system  with  a constant  feedback  gain,  which  does  not  change  even 
if  the  system  changes.  Once  more  this  can  be  deduced  from  another 
1 set  of  coupled  Riccati-like  matrix  difference  equations.  If  they 

diverge  as  the  terminal  time  goes  to  infinity,  then  a constant 
gain  implementation  is  unreliable,  because  it  cannot  stabilize  the 
system.  If,  on  the  other  hand,  there  exists  an  asymptotic  solution 
! to  this  set  of  Riccati-like  equations  then  a reliable  control 

I system  without  feedback  reconfiguration  can  be  obtained.  The 

implementation  requires  constant  gain  state  variable  feedback,  and 
the  feedback  gains  can  be  calculated  off-line. 

In  summary,  these  results  can  be  used  for  off-line  studies 
relating  the  open  loop  dynamics,  required  performance,  actuator 
mean  time  to  failure,  and  functional  or  identical  actuator 
redundancy,  with  and  without  feedback  gain  reconfiguration 
strategies. 
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CHAPTER  1 


INTRODUCTION 


1.1  Motivation  for  the  Research. 

This  report  addresses  some  of  the  current  problems  in  interfacing 
systems  theory  and  reliability,  and  puts  this  research  in  perspective 
with  the  open  questions  in  this  field.  Reliability  is  a relative  concept; 
it  is,  roughly,  the  probability  that  a system  will  perform  according 
to  specifications  for  a given  amount  of  time.  The  motivating  question 
behind  this  report  is:  What  constitutes  a reliable  system? 

Knowledge  of  the  reliability  of  a system  is  crucial.  In  this 
report,  a system  is  reliable  if  it  has  a (quantitative)  reliability  of 
one,  i.e.,  if  the  probability  that  the  system  will  not  perform  according 
to  specifications  for  a given  period  of  time  is  zero.  Therefore,  the 
question  "What  constitutes  a reliable  system?"  can  be  restated  as: 

What  are  the  specifications  which  a system  must  meet  in  order  to  be 
reliable? 

A system  is  normally  designed  in  two  stages:  First,  the  components 
are  selected  in  such  a way  as  to  meet  the  reliability  specifications; 
second,  the  control  problem  is  formulated  and  solved  for  that  configura- 
tion of  components.  Although  this  procedure  is  over-simplified,  it 
illustrates  a second  question:  Should  the  control  problem  influence  the 
choice  of  the  configuration,  and  if  so,  how  can  this  be  achieved?  The 
first  part  of  the  question  is  answered  by  history:  The  control  problem 
influences  configuration  design  now  by  iteration  between  t)\e  two  stages 
of  design.  This  is  most  likely  not  the  beat  method!  If  a theory  wore 
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available  which  allowed  a comparison  between  alternate  designs,  based 
on  both  the  exiiected  system  reliability  and  the  expected  system  perfor- 
mance, it  wovtld  greatly  simplify  the  current  design  methodology.  It  is 
unfortunate  that  at  present  there  is  no  accepted  methodology  for  a 
determination  of  expected  system  performance  which  accounts  for  changes 
in  the  performance  characteristics  due  to  failure,  repair  or  reconfigura- 
tion of  system  functions.  This  report  presents  such  a methodology  for  a 
specific  class  of  linear  systems  with  quadratic  cost  criteria. 

1 . 2 General  Nature  of  the  Problem. 

This  Section  presents  the  general  theoretical  framework  necessary  to 
approach  the  problem  of  reliable  control  system  design.  First,  a 
discussion  of  some  of  the  concepts  in  reliability  theory  will  be  present- 
ed. The  control-theoretic  framework  for  the  specific  topics  covered  in 
this  report  will  then  be  developed.  Finally,  the  interrelationships 
between  systems  theory  and  reliability  theory  will  be  explored,  leading 
to  a mathematical  formulation  of  the  reliable  control  system  design 
problem  and  a discussion  of  the  general  nature  of  the  results  presented 
in  the  remainder  of  this  report. 

1.2.1  Reliability  Theory. 

The  generally  accepted  definition  of  reliability  is  stated  in 
Appendix  1.  Basically,  the  reliability  of  a system  is  the  probability 
that  the  system  will  perform  according  to  specifications  for  a given 
amount  of  time.  In  a system-theoretic  context,  the  specification  which 
a system  must  meet  is  stability;  also,  since,  at  least  for  most  mathemati- 
cal models  of  systems,  stability  is  a long-term  attribute  of  the  system. 
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tlie  amount  of  time  for  which  the  system  must  remain  stable  is  taken  to 
be  infinite.  Therefore,  the  following  definitions  of  system  reliability 
are  used  in  this  report: 

Definition  1:  A system  (implying  the  hardware  configuration,  or  mathe- 
matical model  of  that  configuration,  and  its  associated  control  and 
estimation  structure)  has  reliability  r where  r is  the  probability  that 
the  system  will  be  stable  for  all  time. 

Definition  2:  A system  is  said  to  be  reliable  if  r = 1. 

Definition  3:  A system  design,  or  configuration,  is  reliable  if  it 
is  stabilizable  with  probability  one. 

These  definitions  of  reliability  depend  on  the  definition  of  stability, 
and  for  systems  which  can  have  more  than  one  mode  of  operation,  stability 
is  not  that  easy  to  determine.  In  this  report,  stability  will  mean 
either  mean-square  stability  (over  some  random  space  which  will  be  left 
unspecified  for  the  moment) , or  cost-stability  (again,  an  expectation 
over  a certain  random  space),  which  is  basically  the  property  that  the 
accumulated  cost  of  system  operation  is  bounded  with  probability  one. 

(The  definition  of  cost  is  also  deferred.) 

The  reliability  of  a system  will  depend  on  the  reliabilities  of  its 
various  components  and  on  their  interconnections.  Thus,  the  systems 
engineer  must  have  an  understanding  of  the  probabilistic  mechanisms  of 
component  failure,  repair,  and  system  reconfiguration.  There  are  a 
multitude  of  models  which  can  be  used  for  component  failure  and  repair. 


and  reconfiguration.  IVo  good  references  to  the  mechanics  of  reliability 
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starting  at  the  present  time  is  independent  of  how  long  the  device  has 
been  operational.  This  constant  hazard  rate 

z(t)  = c (1.2.3) 

results  in  the  exponential  failure  distribution  shown  in  Figure  1.1. 

The  constant  hazard  rate  is  a close  approximation  to  the  actual  hazard 
rate  of  many  devices.  For  example,  the  transistor  has  a hazard  rate 
similar  to  that  shown  in  Figure  1.2.  This  type  of  function  is  quite 
common  [Shooman,  1].  Early  failures  in  Region  I of  Figure  1.2  are 
failures  during  the  "burning-in"  of  the  device;  they  are  associated  with 
poor  assembly,  defective  materials  and  other  random  fluctuations  in  the 
manufacturing  process.  Failures  in  Region  III  are  due  to  the  wearing  out 
of  elements  in  the  part.  Region  II  is  relatively  constant  and  closely 
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approximates  the  constant  hazard  rate  function.  In  a large  system,  parts 
are  generally  "burned-in"  before  assembly  is  completed;  therefore,  the 
system  begins  operation  in  Region  II.  As  the  system  ages,  periodic 
maintenance  removes  old  parts  before  the  hazard  rate  rises  in  Region  III. 
Therefore,  the  assumi>tion  of  a constant  hazard  rate  is  usually  justified. 
In  this  report,  the  constant  hazard  rate  function  is  used  exclusively. 

This  is  due  not  only  to  its  broad  applicability,  but  also  to  the  fact  that 
any  non-constant  hazard  rate  requires  a reliable  control  system  to  keep 
track  of  tlie  starting  times  of  the  system's  mode  of  operation. 

I/I  the  discrete-time  case,  to  which  this  report  is  confined  exclu- 
sively, the  hazard  rate  becomes  the  probability  of  failure  (or  repair  or 
reconfiguration)  between  time  t and  time  t+1.  For  a system  with  many 
operating  modes,  the  probability  of  being  in  a given  mode  at  a given 
time,  given  some  past  probability  vector  over  the  various  operating 


modes,  can  be  modeled  by  a Markov  chain.  If  is  a vector 


L+1 


(1.2.4) 


where  there  are  L+1  operating  modes,  then  ti^  is  propogated  in  time  by 


IT  , = P IT 

-t  + 1 1 


where 


(P.j)  e R 


L+1  X L+1 


(1,2.5) 


(1.2.6) 


and 


p. . = prob.  of  system  being  in  mode  i at  time  t+1,  given  it 


1] 


was  in  mode  j at  time  t 


(1.2.7) 


(see  (Paz,  3)).  The  probability  p^^  is  the  discrete-time  equivalent  of 
the  hazard  rate,  and  is  time-invariant.  In  the  future,  a timo-invari- 


i 


ant  Markov  chain  will  be  assumed  as  a model  of  the  modes  of  operation 
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and  the  statistics  of  the  random  switchings  between  modes. 

It  is  now  necessary  to  define  precisely  these  modes  of  operation 
dtuJ  their  dynamic  transitions.  The  terms  system  configuration  and 
system  structure  will  be  used. 

Definition  4;  System  Structure ; A i^jssible  mode  of  operation  for  a 
given  system;  the  components,  their  interconnections,  and  the  informa- 
tion flow  in  the  system  at  a given  time. 

Definition  5;  System  Configuration;  The  original  design  of  the  system, 
accounting  for  all  modeled  modes  of  operation,  and  the  Markov  chain 
governing  the  conf iguration,  or  structural , dynamics  (transitions  among 
the  various  structures) . 


An  example  of  three  possible  structures  for  a given  system  is  shown 
graphically  in  Figure  1.3.  In  this  rei^ort , structui as  are  referenced  by 
convention  by  the  set  of  non-negative  integers 

I - {0,1,2,3, . . . ,l}  (1.2.8) 

The  configuration  for  the  design  illustrated  in  Figure  1.3  is  depnctod 
graphically  in  Figure  1.4.  The  nodes  of  the  graph  in  Figure  1.4 
represent  the  system  structures  of  Figure  1.3.  The  edges  of  the  grap); 
represent  probabilities  of  transfer  from  one  node  to  another,  and  are 
elements  of  the  matrix  P. 


^i+1. j+1 


prob.  structure  i at  time  t+1  given  structure  j at 
time  t. 

(1.2.9) 


The  state  of  the  system  configuration  at  time  t is  the  structure  in 
which  the  system  is  operating  at  that  time. 
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comp.  1 


comp.  3 


structure  1 


comp.  2 


Figure  1.3:  Three  hypothetical  system  structures. 
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k(t)  = structural  state  at  time  t 
k(t)  C I 


(1.2.10) 

(1.2.11) 


This  structural  state  evolves  in  time  to  form  t)ie  structural  t ra jectory 
(of  lenqth  T+1) 

. . ,k(T))  (1.2.12) 

In  qeneral,  tliis  structural  trajectory  is  a random  variable  with  apriori 
probability  of  occurance 


= (k(0),k(l). 


I>(x^) 

(Ficjure  1.5) 


^k(O)  (l)k  (0)^\(2)k(l) 


k (T) k (T-1) 


(1.2.13) 


1.2.2  Control  Theory. 

In  this  report,  only  linear  systi'ms  witl)  a quadratic  cost  index 
are  considered.  At  tliis  time,  any  more  qeneral  formulation  is  of  dubious 
value  in  that  the  linear  quadratic  problem.s  can  demonstrate  many  of  tlie 
fundamental  concepts  of  reliable  control  system  desiqn.  It  is 
doubtful  that  any  otlier  formulation  could  bo  solved  without  the  knowledqe 
qained  from  the  linear  quadratic  solutions  presented  in  the  remainder  of 
this  report.  As  a furtlier  restriction,  perfect  observation  of  t)u'  system 
state  is  assumed.  The  general  class  of  linear  systems  discussed  in 


this  report  is  of  the  form 


-t  + 1 — k(t)— t — k(t)— t 


(1.2.14) 


The  set  of  pairs  (A^^  ) describe  the  possible  system  structures, 

where 

k(t)  C I (1.2.15) 

Tile  remainder  of  tlie  configuration  is  specified  by  t)ie  Markov  cliain 
equation  (1.2.5).  Tlie  objective  of  this  rosearcli  is  to  develop  control 
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laws  which  account  for  the  possible  structural  trajectories  (1.2.12) 

while  minimiziiuj  some  function  of  the  cost.  The  cost  function  for  a 

T-1 

qiven  random  state  and  control  trajectory  ((x^  ,u^  — 


T 


x^Qx^  +u  Ru^  + X Qx 
— t^— t — t t 


(1.2.16) 


Tlie  function  of  the  cost  which  is  minimized  is  generally  taken  to  be  the 
expected  value  of  J^,  over  all  possible  structural  trajectories  x^.  It 
is  shown  t)\at  this  class  of  optimization  problems  yields  solutions 
wliicli  are  .sensitive  to  both  system  performance  and  system  reliability, 
as  motleled  in  the  configuration. 

In  the  remainder  of  the  report,  only  variations  in  t)ie  B-matrix, 
or  actuators  are  considered.  An  actuator  is  a device  w)uch  transfers 
the  control  input  to  the  system  dynamics.  The  actuator  in  the  B-matrix 
may  model  a physical  linkage,  such  as  is  fcmd  on  t)jo  control  surfaces  of 
aircraft,  or,  for  example,  the  effectiveness  of  a tax  reduction  on  the 
economy.  A single  actuator  may  fail  in  many  different  modes.  For 
example,  the  B-matrlx  can  be  of  the  form 

5.0  = Iby  I b J • • • I bj  1 (1.2.17) 

where  the  's  are  actuators  which  may  fail  to  an  actuator  having  zero 
gain  with  a failure  probaility  per  unit  time  p^: 

b . ► 0 (1.2. IH) 

— 1 — 

Then  the  system  structures  representing  modes  of  failure  would  bo  modeled 
as  B-matrices  l\avin<j  at  least  one  zero  column. 

This  class  of  linear  models  can  also  bo  used  as  a model  for  self- 
reorganizing  systems;  the  only  restriction  is  that  the  reorganization, 
or  reconfiguration,  process  niu.s  t be  modeled  witli  a constant  hazard  rate. 
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An  important  aspect  of  this  research  is  the  study  of  various  types 
of  redundancy.  At  present,  the  effect  of  redundancy  on  system  performance 
is  poorly  understood.  There  are  two  basic  types  of  redundancy:  component 
redundancy  and  functional  redundancy.  Component  redundancy  is  the  use 
of  two  or  more  identical  components  (in  this  report,  actuators)  for  the 
same  task.  A good  example  is  provided  by  equation  (1.2.17).  Suppose 
two  actuators,  b^  and  b^  , are  identical.  If  b^  fails  (Equation  (1.2.18)), 
b^  is  still  operational,  and  vice-versa.  In  order  to  lose  the  function 

of  actuators  b . and  b . , both  actuators  must  fail;  this  event  will  . 

-1  -D 

have  a lower  probability  of  occurance  than  the  event  of  the  failure 

of  b^  ; not  in  the  configuration  the  function  of  actuator  j 

b . would  be  lost. 

— 1 

The  problem  with  component  redundancy  in  control  theory  is  how 
should  the  allocation  of  control  resources  t)e  allocated  to  the  redun- 
dant components,  and  how  should  the  component  reliabilities  affect  the 
choice  of  an  optimal  control  law?  The  control  methodologies  presented 
in  this  report  answer  the  question  for  a specific  class  system  confi- 
gurations. 

Functional  redundancy  implies  the  overlapping  of  function  of  two 
or  more  components  in  a system.  If  one  of  the  components  fails,  part 
of  its  function  is  still  performed  by  the  other  (redundant)  component (s) . 

Functionally  redundant  actuators  are  modeled  in  this  report  in  the  same 
way  as  component  redundancy.  The  functional  redundancy  is  accounted  for 
in  the  expectaion  of  the  cost  index  over  the  structural  trajectories. 

The  dynaunics  of  repair  and  reconfiguration  are  all  modeled  in  this 


report  as  exponential  failure  distributions  (constant  hazard  rates) . 
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As  an  example,  if  two  actuators  (b^  and  b ) are  in  a system  configura- 
tion and  can  each  fail  with  probability  p and  p.  per  unit  time, 
respectively,  to  an  actuator  with  zero  gain  (0) , then  the  configuration 


dyncunics  are,  assuming  independence  of  failures: 


B 1 = f 0 I b ^ J 
12  = [bJO] 

B = l£  I 0] 


(1.2.19) 


(1.2.20) 


(1.2.21) 


(1.2.22) 


->•  with  probability  p (1-p^  ) per  unit  time  (1.2.23) 

^0  ^1 


B -*■  B with  probability  p (l-p_  ) pei  unit  time  (1.2.24) 
° ^ ^1  ^0 


B B with  probability  p p per  unit  time 

03  f^ 


(1.2.25) 


B B with  probability  p per  unit  time 

13 


(1.2.26) 


^ ->■  B with  probability  p.  per  unit  time 

2 3 


(1.2.27) 


From  this  information,  the  Markov  chain  transition  matrix  P can  be  formed: 


1-Pf  -Pf  +Pf  Pf  0 0 

0 1 0 1 

Pf  (1  - Pf  ) 1-Pf  0 

P = 0 1 2 


(1.2.28) 


Pf  (1-Pf  ) 
1 0 


0 1-p, 


Pf  Pf 

0 1 


Pf  Pf 

2 1 


Repair  is  considered  to  be  component  replacement,  and  is  modeled  in  the 
same  manner;  e.g.. 


£ ->  B with  probability  p . p 

0 r^  r^ 


(1.2.29) 


Reconfiguration  is  the  restructuring  of  the  (actuator)  configuration  to 


compensate  for  failure,  and  is  modeled  as 


B ‘ B ^ with  probability 
— 1 — 4 41 


(1.2.30) 


whore  B^  is  a new  actuator  configuration  which  will  be  used  on  reconfi- 
guration after  failure. 

The  methodologies  presented  allow  the  study  of  the  effects  of 
failure,  repair  and  reconfiguration  on  the  optimal  control  of  linear 
systems?  they  yield  a quantitative  analysis  of  the  effectiveness  of  a 
given  system  design,  where  effectiveness  is  a quantity  relating  both 
the  performance  and  the  reliability  of  a configuration  design  (see 
Appendix  1). 


1.2.3  General  Nature  of  Results. 

There  are  three  classes  of  reliable  controller  methodologies: 

I)  Passive  (Robust)  Controller  Design 

II)  Active  (Switching)  Controller,  Passive  Configuration  Design 

III)  Active  Controller,  Active  Configuration  Design 
This  report  concentrates  entirely  on  classes  I)  and  II).  Class  III) 
methodologies  are  much  more  difficult  to  study.  The  Marlcov  chain  models 
of  configuration  dynamics  which  work  in  classes  I)  and  II)  do  not  hold 
in  class  III);  as  yet,  there  is  no  satisfactory  way  to  model  the 
configuration  dynamics  of  a system  in  such  a way  that  the  control  rules 
are  well-defined. 

Class  I)  methodologies  are  passive  designs.  These  designs  account 


for  the  occurance  of  failures  in  t)u'  initial  selection  of  tlie  control 
law;  on-line,  this  class  of  designs  does  not  use  any  current  estimate  of 
the  structural  state  of  the  configuration.  The  design  is  "conservat ive" 
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in  that  it  continues  to  stabilize  the  system  without  regard  to  the  current 
structural  state.  A special  sub-class  of  these  designs  is  the  robust 
controller  designs.  A robust  controller  will  stabilize  any  structure  of 
the  system  without  regard  to  the  conf iguration  dyncunics ; i.e.,  if  the 
system  remains  in  any  structural  state  forever,  it  will  still  be 
stabilized.  The  class  I)  methodologies  are  represented  by  the 
non-switching  gain  methodology  of  Chapter  5. 

Class  II)  methodologies  are  active  controllers;  in  some  sense, 
they  are  adaptive.  From  knowledge  of  the  system's  past,  these  controllers 
switch  their  control  law  on-line  in  order  to  compensate  for  what  they 
estimate  to  be  the  correct  structural  state.  For  deterministic  systems, 
these  controllers  can  be  determined  analytically.  For  stochastic 
systems,  the  optimization  problems  cannot  be  solved  analytically  in 
general  due  to  the  dual  control  effect  [Fel'dbaum,  4-  7).  Thus, 
suboptimal  control  strategies  must  be  used.  The  class  II)  methodologies 
are  represented  by  the  switching  gain  methodology  in  Chai)tor  3 and 
its  suboptimal  extensions  in  Chapter  4. 

\ 
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1.3  Relations  with  Previous  Literature. 


This  research  is  based  on  a background  knowledge  in  both  reliability 
theory  and  systems  theory.  Both  mathematics  and  probability  theory  are 
fundamental  in  these  fields.  As  general  references  to  the  techniques 
used  in  this  report,  in  real  analysis,  and  measure  and  integration 
theory,  lRudin,8l,  (Segal  s Kunze,  9),  and  (Halmos.lO)  are  good;  in 
matrix  theory,  (Gantmacher , ll)  is  the  standard  reference.  In  probabil- 
ity theory,  [Bauer, 12]  and  (Doob,13]  are  definitive;  expansions  on  the 
theory  of  Markov  chains  are  found  in  (Chung, 14)  and  {Derman,151. 

There  are  several  good  texts  on  reliability  theory;  of  these, 

(Greene  & Bourne,  2J  and  (Shooman,  1)  are  possibly  the  best.  (Cox, 16) 
and  (Corcoran, 17]  demonstrate  the  current  methods  of  the  scheduling  and 
use  of  redundancy  in  reliability  technology.  Other  good  treatments  are 
found  in  (Barlow  and  Proschan,18]  and  (Gnedenko, 19] . 

In  control  theory,  a good  treatment  of  the  deterministic  linear 
quadratic  regulator  problem  is  found  in  the  IEEE  Transactions  Special 
Issue  edited  by  (Athans,20J,  and  in  (Athans  & Palb,21).  The  dual 
control  problem  is  described  in  [Fel'dbaum,  4-  7]  and  several  other 
publications. 

Previously,  several  authors  have  studied  the  optimal  control  of 
systems  with  randomly  varying  structure.  Most  notable  among  these  is 
(Wonham,22] , where  the  solution  to  the  continuous  time  linear  regulator 
problem  with  randomly  jumping  parauneters  is  developed.  This  solution  is 
similar  to  the  discrete  time  switching  gain  solution  presented  in 
Chapter  3.  The  random  parameters  are  restricted  to  be  a continuous 
time  Markov  chain.  The  most  notable  difference  is  that  in  (Woniiam, 22] , 


30 


the  assumption  is  made  that  the  controller  has  perfect  information  about 
the  present  state  of  the  random  process  on-line . The  solution  switches 
ijains  in  a linear  state  feedback  control  law  whenever  the  (Markovian) 
random  parameter  jumps.  In  the  discrete  time  switchiny  yain  solution 
presented  in  Chapter  3,  t)»e  control  law  is  determined  from  past  observa- 
tions wtiicJi  allow  tlie  deduction  of  the  exact  state  of  tl»e  random  para- 
meter process,  and  then  tlie  random  parameter  may  switch  values  according 
to  the  statistics  given  by  tl\o  Markov  chain.  Tlius,  tlie  control  may  be 
applied  to  one  of  a number  of  possible  structures  at  the  next  time 
instant.  In  Wonham's  development,  the  optimal  control  law  is  matched 
specifically  to  one  structure.  The  analogous  continuous  time  version 
to  the  switching  gain  solution  of  Chapter  3 would  be  to  assume  on-line 
perfect  observation  of  the  randtim  parameter  with  a fixed  time  delay. 
Wonham's  result  has  no  such  time  delay. 

Wonham  also  proves  an  existence  result  for  the  steady-state  optimal 
solution  to  the  control  of  systems  with  randomly  varying  structure. 

This  result  is  based  on  conditions  of  stabilizability  of  each  system 
structure  and  observability  of  each  structure  with  respect  to  the 
cost  functional . Tlie  conclusion  is  only  sufficient;  it  is  not  necessary 
for  existence  of  a steady-state  solution.  Similar  results  were  obtained 
in  (Beard, 23]  for  the  existence  of  a stabilizing  gain,  whore  the 
structures  were  of  a highly  specific  form;  tliese  results  were  necessary 
and  sufficient  algebraic  conditions,  but  cannot  be  readily  generalized 
to  less  specific  classes  of  problems. 

T(u>  time-varying  solution  of  |Won(tam,22)  is  i-omputed  using  a set  ol 


coupled  Riccati-like  matrix  equations.  Tlie  coupling  is  ii\  the  torm  of 
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a linear  term  in  the  solution  to  the  matrix  equations  added  to  the  normal 
linear  quadratic  Riccati  equation.  The  solution  can  be  precomputed  by 
solving  the  coupled  Riccati-like  equations  off-line;  the  control  law  is 
then  switched  on-line  to  a gain  which  corresponds  to  the  current  state 
of  the  Markov  process.  The  optimal  solution  requires  perfect  knowledge 
of  the  structure. 

In  reality,  the  structure  is  seldom  known  perfectly,  and  a noisy 
observation  of  the  random  process  leads  to  a dual  control  problem. 
Although  much  of  Chapter  3 is  based  on  the  fact  that  the  controller  can 
obtain  the  structural  state  with  one-step  delay  in  the  deterministic 
discrete  time  problem,  this  report  makes  the  connection,  for  the  first 
time,  of  the  existence  of  a steady-state  switching  gain  controller  with 
that  system's  reliability  and  effectiveness. 

(Sworder, 24]  has  developed,  using  a version  of  the  stochastic 
maximum  principle,  an  optimal  feedback  control  law  for  a class  of  linear 
systems  with  jump  parameters  which  is  almost  identical  to  that  of 
^cnham, 22];  the  coupled  Riccati-like  equations  are  identical  except  for 
notation.  The  only  difference  is  Sworder 's  assumption  that  the  random 
process  is  instantaneously  observable  from  a set  of  sensors  which  are 
unaffected  by  the  choice  of  the  control  law.  Using  this  assumption, 
Sworder  avoids  the  problems  of  dual  control. 

Sworder  also  comments  on  the  usefulness  of  linear  system  models 
with  jump  parcuneters  in  modeling  possible  failures  in  the  system 
[Sworder ,24 ] . [Ratner  & Luenberger ,25 ] derive  a control  law  for  a 
continuous  time  linear  system.  The  system  has  one  failure  mode,  and  a 
maximum  number  of  renewals  (repairs)  can  take  place.  The  objective  is 
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to  determine  apriori  the  optimal  time  intervals  in  which  the  system 
should  operate  in  the  failure  mode,  and  the  optimal  control  law,  ijiven 
the  mode  of  operation,  over  a finite  time  interval.  The  failure  process 
is  assumed  to  have  an  exponential  failure  distribution  (constant  hazard 

1 

rate);  the  renewal  process  is  controlled,  and  is  not  random.  The 
control  law  is  of  the  switching  gain  type,  and  the  solution  is  in  the 
form  of  two  coupled  Riccati-like  matrix  equations  quite  similar  to  those 
in  [Wonham,22]  and  (Sworder ,24] . The  optimal  control  policy  and  the 
optimal  renewal  policy  can  both  be  calculated  off-line.  This  class  of 
problems  is  further  investigated  by  [Sworder, 26)  to  determine  over  what 
region  immediate  renewal  is  the  optimal  policy.  Both  of  these  papers 
illustrate  examples  of  class  III)  control  methodologies;  the  structural 
state  as  well  as  the  system  state  is  under  the  influence  of  the  control- 
ler. The  simple  structure  of  the  class  of  systems  studied  by  [Ratner  & 

Luenberger ,25 1 allows  a solution.  There  is  need  for  much  more  work  in 
this  area. 

Still  a third  approach  to  the  problems  associated  with  multiple-  — 

structure  systems  is  given  in  [Bar-Shalom  & Sivan,27].  Here,  the 
measurements  of  the  system  state  are  corrupted  by  additive  noise.  The 
open-loop  controller  and  the  open-loop  feedback  controller  are  derived 
using  dynamic  programming.  Knowledge  of  the  present  state  of  the  random 
process  governing  the  system  configuration  is  not  assumed.  Therefore, 
the  (optimal)  closed-loop  controller  would  be  a dual  control  law.  The 
open-loop  controller  assumes  no  on-line  measurements  of  the  system  state; 

the  open-loop  feedback  controller  assumes  future  on-line  measurements  1 


and  thereby  improves  its  performance.  There  is  little  correlation 
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b<.?tween  this  paper  and  the  research  on  which  this  report  is  based. 

(Winner, 28]  developed  a suboptimal  control  scheme,  which  allowed 
for  imperfect  observation  of  the  random  parameter  process,  known  as 
multiple-model  adai>tive  control.  In  this  method,  the  parameters  could 
only  take  a discrete  set  of  values,  a cause  of  recent  disfavor,  as  MMAC 
does  not  always  work  well  when  the  parameters  vary  continuously  and  are 
approximated  by  the  mathematics.  Similar  work  has  been  done  in  [Pierce  & 
Sworder,29] . The  MMAC  methodology  is  optimal  one  step  backward  from  the 
final  time,  as  is  the  switching  gain  methodology  in  the  example  of 
Chapter  2 when  applied  to  systems  with  additive  white  control  noise. 

The  dual  problem  of  state  estimation  with  a system  with  random 
parameter  variations  over  a finite  set  was  studied  in  [Chang  & Athans,30]. 
It  is  shown  there  that  the  optimal  estimator  consists  of  a geometrically 

i 

1, 

increasing  set  of  Kalman  filters,  one  for  each  possible  structural 

trajectory  of  length  t+1  at  time  t,  and  an  averaging  process  to  compute 

the  minimum  mean-square  error  estimate  from  the  filter  estimates.  It 

is  also  shown  that  when  the  parameter  process  is  Markovian,  a bank  of 
2 

N estimators  is  optimal,  where  there  are  N possible  values  of  the 
parameters.  Each  estimator  is  then  conditioned  on  the  possible  values 
of  the  parameters  at  the  two  previous  time  instants. 

Recently,  the  robustness  of  the  linear  quadratic  regulator  has  been 
studied  in  depth.  This  work  is  described  in  (Wong,  ejt.  ^.,31]  and 
in  [Safonov  & Athans,32].  A long-standing  problem  with  the  linear 
quadratic  design  methodology  has  been  the  lack  of  analogs  to  tlie  various 
stability  and  robustness  criteria  of  classical  systems  theory.  This 
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research  was  aimed  at  characterizations  of  robust  solutions  to. 
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specifically,  the  linear  quadratic  regulator.  Supporting  research  is 
reported  in  (Safonov  & Athans,33],  (Wong  & Athans,34],  (Wong,35J,  and 
(Safonov, 36 J . The  research  in  this  report  is  related  to  tlie  robust 
controller  problem,  but  the  approach  is  different  in  that  the  performance 
criterion  is  modified  to  account  for  possible  variations  in  structure, 
such  as  those  caused  by  failures,  rather  than  depending  on  certain 
properties  of  the  linear  quadratic  regulator  solution  to  guarantee 
robustness.  In  this  research,  the  concept  of  stability  is  related  to 
the  existence  of  a finite  cost  solution  to  the  non-switching  gain 
problem.  For  a specific  class  of  configurations,  this  approach  solves 
the  robust  controller  problem  (Chapter  5,  Section  9) . 

The  existence  of  an  uncertainty  threshold  for  the  non-switching 
controller  of  Chapter  5,  that  limit  on  parameter  uncertainty  beyond 
which  no  controller  can  stabilize  the  system,  is  proven  for  an  one- 
dimensional example.  This  work  is  similar  to  the  work  by  (Athans, 
et.  a^.  ,37]  on  the  Uncertainty  Threshold  Principle  and  the  related 
papers  by  [Ku  & Athans, 38]  and  (Ku,  et.  ^.,39].  This  research  is 
reported  in  Chapter  2,  Section  7. 

Lastly,  parts  of  this  research  have  been  presented  in  an  unpub- 
lished form  at  the  1977  Joint  Automatic  Control  Conference  in  San 
Francisco,  and  published  for  the  1977  IEEE  Conference  on  Decision  and 
Control  Theory  in  New  Orleans  (Birdwell  & Athans,40]. 
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1.4  Summary  of  Main  Contributions . 

There  are  two  major  contributions  of  this  research.  First,  the 

I classification  of  a system  desiyn  as  reliable  or  unreliable , for  the 

i 

deterministic  variable  actuator  linear  system  in  Chapter  3,  has  been 
equated  with  the  existence  of  a steady-state  switching  gain  and  cost 
for  that  design.  If  the  steady-state  switching  gain  does  not  exist, 
then  the  system  design  cannot  be  stabilized;  hence,  it  is  unreliable . 

The  only  recourse  in  such  a case  is  to  use  more  reliable  components 
and/or  more  redundancy.  Reliability  of  a system  design  can  therefore 
be  determined  by  a test  for  convergence  of  the  set  of  coupled  Riccati- 
like  equations  (3.3.6)  as  the  final  time  goes  to  infinity. 

A similar  result  holds  for  the  non-switching  gain  methodology  of 
Chapter  5.  Here,  the  system  design  is  classified  as  reliable  or 
unreliable  with  respect  to  a constant  gain  linear  feedback  control  law, 
depending  on  the  convergence,  or  divergence,  respectively,  of  equation 
(5.6.16)  as  the  final  time  goes  to  infinity.  If  equation  (5.6.16) 
converges  to  a limit  cycle,  then  that  limit  cycle  produces  a stabilizing 
cyclic  steady-state  gain. 

The  second  major  contribution  lies  in  the  robustness  implications 
of  the  non-switching  gain  methodology.  Precisely,  a constant  gain  for 
a linear  feedback  control  law  for  a set  of  linear  systems  is  said  to 
be  robust  if  that  gain  stabilizes  each  linear  system  individually , i.e., 
without  regard  to  the  configuration  dyneunics.  The  problem  of  determining 
when  such  a gain  exists,  and  of  finding  a robust  gain,  can  be  formulated 
in  the  context  of  the  non-switching  gain  methodology.  As  a result,  the 
non-switching  gain  methodology  gives  an  algorithm  for  determining  a 
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robust  gain  for  a set  of  linear  systems  which  is  oiitimal  with  respect  to 
a cjuadratic  cost  criterion.  If  the  algoritlun  does  not  converge,  then 
no  robust  gain  exists. 

The  following  Section  of  this  Chapter  will  outline  the  remainder 
of  this  report. 

1.5  Outline  of  Report . 

In  Chapter  2,  several  one-dimensional  examples  are  examined  as 
a clarification  and  motivation  for  the  methodologies  presented  in 
Chapters  3 through  5.  In  addition,  Chapter  2,  Section  7,  deals  with 
the  relationship  between  the  Uncertainty  Threshold  Principle  and  the 
existence  of  a steady-state  solution  to  the  non-switching  gain  problem. 

Chapter  3 develops  the  optimal  solution  to  the  class  of  problems 
described  in  Section  2 of  this  Chapter.  The  solution  is  labeled  the 
switching  gain  solution  because  the  gain  of  a linear  feedback  control 
law  switches  in  response  to  the  exact  observation  of  the  system 
structure  with  one-step  delay. 

Since  Chapter  3 deals  entirely  with  deterministic  systems,  and  the 
switching  gain  solution  does  not  extend  optimally  to  the  stochastic 
case.  Chapter  4 presents  some  suboptimal  methods  which  can  be  used  to 
extend  the  switching  gain  solution  to  stochastic  problems.  Two 
methodologies  are  presented.  One  (hypothesis  testing)  is  based  entirely 
on  estimation  of  the  structure.  The  st'cond  (dual  identification)  uses 
the  dual  effect  of  the  control  law  to  determine  more  precisely  wliat  the 
structure  is  with  the  next  observation.  The  optimal  control  law  would 
have  some  characteristics  of  both  methodologies,  as  is  shown  by  example 
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in  Chapter  2,  Section  5. 

Chapter  5 derives  a control  law  which  ignores  any  on-line  informa- 
tion which  might  be  gathered  about  the  structural  state,  and  results 
in  a non-switching  gain  solution  used  in  a linear  feedback  control  law. 
The  stability  of  this  non-switching  solution  is  explored,  along  with 
the  existence  of  a steady-state  solution,  in  Secion  7.  In  Section  9, 
the  robustness  issue  is  addressed,  and  the  non-switching  methodology  is 
used  to  define  an  algorithm  which  can  determine  the  existence  of  a 
robust  gain  and  calculate  an  optimal  robust  gain  with  respect  to  a 
quadratic  cost  functional,  when  one  exists. 

Chapter  6 focuses  on  the  issues  of  computer-aided  design  and  the 
application  of  the  non-switching  gain  methodology  to  design  problems. 
Two  examples  are  used  to  demonstrate  the  effectiveness  of  the  non- 
switching methodology  in  design. 

Chapter  7 reviews  the  results  described  in  the  report  and  suggests 
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CHAPTER  2 

CUVRIEtCATION  AND  MOTIVATION  OF  RESEARCH 

2 . 1 Introduction. 

The  purpose  of  this  Chapter  is  to  motivate  all  subse'iuent  more 
genorai  Chapters  with  simple  one-dimensional  examples.  In  particular, 
in  Section  2,  a one-dimensional  problem  is  formulated  and  solveil  to 
illustrate  the  optimal  (switching  gain)  deterministic  control  for 
linear  quadratic  systems  with  variable  actuator  configurations. 

Tile  effects  of  process  noise  on  this  solution  are  examined  in 
Section  3.  The  dual  effects  which  occur  in  the  stochastic  systems 
motivate  the  suboptimal  approaches  described  in  Chapter  4. 

The  possibility  of  steady-state  control  of  variable  actuator 
configuration  systems  with  a single  linear  independent  control  law 
is  discussed  in  Section  6,  motivating  the  work  on  the  non-switching 
gain  solution  and  robust  control  laws  in  Chapter  5.  In  addition, 
the  possibility  of  existence  of  a steady-state  stabilizing  lineai 
feedback  control  law  with  constant  gain  is  compared  witli  tlie  work  on 
the  Uncertainty  Threshold  Principle  (Athans.e^.cU. , 37)  in  Section  7. 
Section  7 contains  the  only  case  of  this  report  wheie  exact  algebraic 
conditions  for  the  existence  of  a steady-state  solution  have  been 
derived.  Unfortunately,  those  results  do  not  readily  extend  in  an 
analytical  manner  to  higher  dimensions. 

The  question  of  existence  of  a steady-state  solution  to  tliese 
problems  is  of  great  importance.  A system  design  is  defined  to  bi- 
reliable  wit)i  respt'ct  to  a certain  class  of  control  laws  if  tt\ere 
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exists  a control  law  from  that  class  for  which  the  infinite  time 
cost  incurred  using  that  control  law  is  finite.  Since  the  switching 
and  non-switching  gain  solutions  are  the  optimal  solutions  for  their 
respective  classes  of  control  laws,  if  they  incur  an  infinite  cost,  so 
will  any  other  control  law  from  that  class.  In  addition,  since  the 
switching  gain  solution  is  the  optimal  control  law  for  the  determin- 
istic problem,  a system  design  is  termed  deterministically  reliable , 
or  reliable  if  and  only  if  the  incurred  infinite  time  expected  cost 
is  finite. 

In  the  next  Section,  a one-dimensional  example  is  presented 
which  will  be  used  to  motivate  the  remainder  of  this  report  by 
examining  the  ramifications  of  the  switching  and  non-switching  gain 
solutions  through  their  specific  application  to  the  example. 

2.2  A Simple  Example — The  Optimal  Solution. 

The  following  one-dimensional  example  is  used  to  demonstrate  the 
switching  gain  methodology  presented  in  Chapter  3,  and  to  show  that 
the  general  stochastic  problem  is  analytically  intractable.  All  proofs 
and  derivations  are  given  in  Appendix  2. 


2.2.1  Problem  Statement. 

L^t  the  discrete-time  system  be  one-dimensional  with  one  control 
variable  u^  and  state  variable  x^  related  by 

Vl  " ^’‘t  " Vt  <2.2.1) 

The  value  of  the  control  multiplier  (bj^'  is  a random  variable  which 
talces  on  one  of  two  discrete  values  at  each  time  t. 
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b if  k = 0 

b (t)  = (2.2.2) 

I 1/b  if  k = 1 

The  random  process  k(t)  is  governed  by  the  Markov  chain  represented 


by 


II  = P IT 

-t  + 1 1 


where 


-t  ^ ^ 


r = 


Pll  ^12 


^21  ^22 


(2.2.3) 

(2.2.4) 

(2.2.5) 


At  any  given  time  t,  the  following  sequence  of  events  occurs: 

I)  X is  observed  exactly,  b computed,  and  k(t-l)is 

t K \ X J 

set  to  0 or  1 depending  on  # where  k(t-l)is  the 

variable  representing  the  Markov  chain; 

II)  b may  change  values  to  b ; 

k ( X ) k ( t ) 

III)  u^  is  applied. 


For  any  given  sample  path,  the  performance  index  is  given  by 

T 

J = ^ (qx^  + ru^) 


(2.2.6) 


where  {o,1,...,t}  is  the  time  set  over  which  the  system  is  to  be 
controlled.  The  objective  of  the  control  problem  is  to  minimize  tlie 
expected  cost-to-go  at  time  t,  given  by 


V(x^,k(t-1) , 


2 2 

u^,t)  = E > (qx^  + ru^ 


) k(t-l) 


(2.2.7) 


where  the  expectation  is  taken  over  all  possible  sample  paths  of 
k(T) , t < T < T. 
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2.2.2  Summary  of  Solution. 

From  Appendix  2.1,  we  find  that  the  optimal  control  is  given 


’o,t  * "l,t 

■■  * "o.t  '‘\,t  * ’l.t 


(2.2.8) 


where 


^it-i 


(2.2.9) 


Thus,  the  control  law  is  linear  in  the  state  x^,  and  switches  between 
two  precomputable  gains,  depending  on  the  value  of  )t(t-l). 

Given  x^,  x^  and  u^ 


^t-l 


1 

if  = b 

0 Vi 


(2.2.10) 


= 1/b 


and  k(t-l)  = 0 if  01  ' or  1 if  TL^_j=(0  11*. 

The  optimal  cost-to-go  is 


VMx^,)c=i,t)  = x^S.^^ 


(2.2.11) 


whore  S and  S are  propagated  backward  in  time  by  the  following 

Oft  1 # t 

equations : 
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Assuming  k=0  at  time  t,  then  tt  ^ ^21^' 


'’'a"  '^"Pll^\,t.l"P21<l/^'>"l.t.l  / 


P 1 1^^^0,t-t-l'^P21^^^^^^1,t+l 


■^^211®'  2 ,J1^  I ^l,t+l  (2.2.12) 

\ btr+Pj^j^b  Sq  / 


Assuming  )c=l  at  time  t,  then  = (Pj^2  ^22^  ' 


^lt~*^^  2 2 2 

f""Pl2‘’  "0,t.l"P22<l/^  >"l.t.ll 


^ Pl2  r ■ 


^ P22  r ■ 


b[Pi2abSo^tH.i'*'P22^^^^^^1,t+l^ 


Pl2^^^0,t^l^P22<^-^^>^l,t+l 

‘’f^^Pl2^^®0,t+l^P22^1,t+l/^^^ 


0,t+l 


l,t+l  (2.2.13) 


Note  from  equation  (2.2.8)  that  u^  switches  from  one  linear  gain 
to  another,  depending  on  the  value  of  x^  — thus,  this  solution  depends 
on  an  exact  Icnowledge  of  x^.  If  knowledge  of  x^  is  corrupted  by  measure- 
ment noise  (or,  if  u^  is  corrupted  by  control  noise) , then  it  will  be 
shown  by  example  that  this  becomes  a dual  control  problem. 
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2.3  The  Dual  Control  Effect. 

To  demonstrate  the  difficulties  encountered  when  white  process 
noise  is  present,  the  optimal  solution  for  the  one  dimensional 
example  is  derived  over  the  time  interval  {0,1,2}  with  additive  white 
control  noise  present.  The  system  is  now  represented  by 

Vl  = ^’‘t  ^ \(t)“t  * ^t  ‘2.3.1) 

is  discrete  time  white  noise  with  zero  mean,  E[f..C  ) = E6  , 
t t—T 

probability  distribution  p(^),  and  is  uncorrelated  with  and  k(T) 
for  t. 

* * 

Thus,  the  problem  is  to  find  u^  and  u^^  such  that  the  expected 
cost-to-go  is  minimized. 

From  Appendix  2.2,  the  optimal  control  one  step  back  in  time 


(at  t=l)  is 


“l  = - 


1^  TT.  (l|l)b.  I qa 

r + TT^(l|l)b2  q 


(2.3.2) 


where  is  the  probability  that  = i,  given  the  information 

set  ,Xq,Uq,Xj^}.  As  expected,  this  control  is  of  the  same 

form  as  is  the  deterministic  control  law,  equation  (2.2.8),  since 
there  is  no  benefit  in  trying  to  determine  more  av-:curately  througli 
the  use  of  a special  control  value.  In  other  words,  there  is  no  dual 
control  effect  at  t = T^-1  (in  this  example,  t=l). 

At  t=0,  the  situation  is  different.  Now,  the  optimal  control  will 
force  the  system  to  supply  more  information  tlirough  the  state  at  t=l 
than  it  normally  would  in  the  absence  of  the  process  white  noise 
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In  order  to  compute  Uq,  a numerical  minimization  of  a numerical 

* 

integration  (in  general)  must  be  performed.  Thus,  u^  is  the 
solution  of 


V (Xq,0)  = min 


qd+a*-) 


+ -q 


r + 


dp{xJk^,k^.2Q)p^^^^ 


"o'°) 


(2.3.3) 


where 


n^dll) 


p(Xi-axQ-b.Uo)TT.  0 


P(xraVbiUo)^,o 


(2.3.4) 


and  p(Xj^(kj^,kQ,Zp)  is  the  probability  measure  of  x^^  over  R(Xj^), 
the  range  of  given  k^^,  k^,  and  Z^. 

Equation  (2.3.3)  is  very  difficult  to  solve  numerically,  and 
for  any  realistically-sized  problem  would  be  economically  infeasible. 
For  the  limited  amount  of  computation  that  has  been  done  with  equation 
(2.3.3),  the  dual  control  effect  is  evident  from  Table  2.1.  Note 

that  as  the  process  noise  variance  increases,  the  trend  is  for  the 

* 

control  Uq  to  increase.  This  is  due  to  the  need  for  a larger  control 
to  lessen  the  effect  of  noise  on  future  estimations  of  the  structure. 


I 
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Table  2.1 


The  optimal  control  versus  and 


^0 

Uo  (H=3) 

Uq 

Uq  (H=10) 

-2.0 

2.3170089 

2.3188635 

2.3201611 

-1.6 

* 

1.8550055 

1.8559061 

-1.2 

^ 1.3898305 

0.9255912 

1.3907551 

1.3912676 

-0.8 

0.9259997 

0.9261950 

-0.4 

0.4606236 

0.4606920 

0.4607206 

0.0 

-0.005 

-0.005 

-0.005 

0.4 

-0.4706236 

-0.4706920 

-0.4707206 

0.8 

-0.9355912 

-0.9359997 

-0.9361950 

1.2 

-1.3998305 

-1.4007551 

-1.4012676 

1.6 

-1.8635511 

-1.8650055 

-1.8659061 

* - calculation  did  not  converge  due  to  numerical  errors 

The  system  used  in  the  calculations  is  described  by  equation 
(2.2.1)  where 
a = 2. 

k(t)  is  0 or  1 
bo  = 2. 
bi  = .5 
q = 3. 
r = 1. 

P=  I*" 

- |.3  .71 

1.51 

-0  ° (.51 

Table  2.1  is  only  intended  to  demonstrate  the  difference  in  the 
optimal  control  laws  at  time  0 for  a two-stage  process;  numerical 

accuracy  is  not  assured.  Specifically,  the  values  of  -.005  for 

* 

Uq  (Xq  = 0)  are  highly  doubtful,  as  well  as  the  consistent 


asymmetry  between  positive  and  negative  values  in  the  Table. 


2.3.1  A Special  Case. 

It  is  interesting  that  for  one  specialized  probability  distribu- 

* 

tion  P(^)/  when  the  optimal  control  is  large  enough,  the  optimal 
solution  is  identical  with  the  deterministic  solution  of  Section  2. 
Prom  Appendix  2.3,  assuming 


P(C)  = 


-/35  < Cl  /3E 


(2.3.5) 


0 otherwise 


as  shown  in  Figure  2.1,  if  u^  from  the  deterministic  solution  (equation 


2.2.8)  satisfies 

I (b^^  - b.)u*|  > 2/3H  for  ^ i 
then  Uq  is  also  the  solution  to  the  stochastic  control  problem. 


(2.3.6) 


Physically,  because  the  noise  is  amplitude  limited,  it  is  easy 
to  exactly  deduce  the  structure  if  the  control  is  large  enough. 


Figure  2.1;  A probability  dis 
white  noise. 


48 


2 . 4 Existence  of  a Steady-State  Solution. 

Although,  as  will  be  stated  in  Chapter  3,  little  can  be  said 
about  the  existence  of  a steady-state  solution  to  the  general  n-dim- 
ensional  switching  gain  problem,  for  the  one-dimensional  example, 
exact  conditions  for  the  existence  of  a steady-state  solution  can 
be  found.  They  are  in  the  form  of  two  simultaneous  algebraic  equations 
which  can  be  solved  analytically. 


r = P, 


b[pj^j^ab+P2j^  (a/b)h]  \ 2 

Pj^j^b^+P2j^(l/b^)h  / 


Pj^j^ab+P2j^(a/b)h  v 2 
t>[Pllb^+P2ih/b^]  / 


(2.4.1) 


hr  = p- 


b[Pi2ab+P22(a/b)h] \ 2 


+ P a 

\ bip^jb  tpjjh/b  ) 


(2.4.2) 


The  equations  are  derived  in  Appendix  2.4.  In  these  equations  the 
varicUsles  T and  h are  defined  as 


r = lim 


(2.4.3) 


t-v-co  0,t+l 


h = lim 


(2.4.4) 


t-*-oo  o,t 


whenever  both  S and  S,  increase  without  bound  as  t -»•  -™,  as  defined 
in  equations  (2.2.12)  and  2.2.13). 
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Since  r is  the  limiting  value  of  the  ratio  of  the  next  value  of 

to  the  present  value  it  is  necessary  that 

I'  ± 1 (2.4.5) 

for 


S 


0,t 


CO 


(2.4.6) 


Similarily,  if  S has  a limit,  then  F can  have  a maximum  value  of 
u ^ t 

1.  Therefore,  a test  can  be  made  on  the  solution  (h,r)  to  equations 
(2.4.1)  and  (2.4.2)  for  the  existence  of  a steady-state  solution: 

If 


h 0 or  ® (2.4.7) 

then 


'o.t'  Si,t'^  * if  r > 1 

(2.4.8) 

i , S converge  if  F < 1 

U » u X # t 

(2.4.9) 

and  there  is  no  conclusion  if  F = 1. 

By  way  of  eliminating  all  possibilities,  as  an  aside,  a limit  cycle 
to  the  solution  of  equations  (2.2.12)  and  (2.2.13)  cannot  occur  by 
Lensna  1 of  Chapter  3. 


2.5  Conclusions  on  the  Switching  Gain  Methodology. 

The  purpose  of  the  last  three  Sections  on  the  one-dimensional 
switching  gain  example  was  to  clarify  the  approach  of  this  phase  of  the 
research,  and  to  motivate  the  approach  of  Chapters  3 and  4.  In  this 
Section,  some  implications  of  the  one-dimensional  example  will  be 
discussed. 


I 


f 
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2.5.1  Implications  of  the  Dual  Control  Effect. 

It  was  shown  in  Section  2 that  the  optimal  solution  to  the  deter- 
ministic class  of  variable  actuator  linear  quadratic  control  problems, 
i.e.,  the  switching  gain  solution,  is  conceptually  straightforward, 
although  computationally  complex  off-line.  Unfortunately,  in  Section  3, 
it  was  demonstrated  that  the  optimal  solution  of  the  stochastic  version 
of  the  Scime  problem  is  infeasible.  (Witness  the  problems  of  calculating 
the  two-step  optimal  solution.)  Therefore,  since  the  switching  gain 
deterministic  solution  is  essentially  the  only  solution  which  can  be 
described  analytically,  the  research  involved  in  developing  the 
n-dimensional  switching  gain  solution  is  justified.  This  is  exactly 
what  is  presented  in  Chapter  3. 

It  then  remains  to  investigate  any  extensions  (which  will  of 
necessity  be  suboptimal)  which  may  be  made  to  the  switching  gain 
solution  to  adapt  the  solution  to  the  stochastic  problem.  In  Chapter 
4,  a start  is  made  in  that  direction.  These  are  two  basic  routes 
to  follow:  The  various  hypothesis  testing  algorithms  in  combination 
with  the  switching  gain  solution,  and  a formulation  developed  in 
Chapter  4 which  gives  the  control  vector  a dual  effect;  the  control 
is  changed  to  increase  the  accuracy  of  the  estimation  algorithm. 

The  optimal  control  would  use  techniques  from  both  categories,  as  the 
dual  effect  is  clearly  seen  in  Table  2.1. 

2.5.2  Existence  of  a Steadv-State. 

Although  for  the  one-dimensional  example,  it  is  possible  to 
determine  the  condition  for  convergence  of  the  Riccati-like  equations 


(2.^.1^’)  aiiil  (2.2.1)),  tills  HK'tluxl  vloos  not  oxtcMul  to  t ho  n-dimon- 
sional  solution.  It  is  at  prosont  unknown  uiulor  what  conditions 
the  Kicirat  i- 1 ike  equations  for  the  n-dimensioiial  piobleni  convei  qe; 


I 


2.6  A Simp  It'  Exami^le — The  Non-Switchin<j  Solution . 

tn  the  previous  sections  of  this  Chapter,  ratJtivation  was  given 
for  the  development  of  the  optimal  (switchii\ij)  solution  to  the  liiu'ar 
guailratic  variable  actuator  configuration  contiol  i>roblem. 

Several  problems  with  tlu;  methotl  wert;  [jointed  out  in  Section  ‘j. 
S[jecifically,  the  metl\odology  does  not  extend  oiJtimally  to  tlie  stoclias- 
tic  case  due  to  the  dual  control  effect.  Secondly,  the  increase  in 
on-line  comi>lexity  over  the  usual  linear  quadratic  centred  [>roblem 
is  significant,  especially  in  the  subo}'tim<il  stochastic  schemes. 

In  many  instances,  a stabili;2ing  .'solution  to  this  class  tjf 
control  problems  is  desired  which  exhibits  the  same'  comi>lexity  as 
does  the  usual  linear  quadratic  controller.  Por  instance,  it  may  be- 
desired  that  a control  law  stabilizt'  a system  without  reqviiring 
error  detection  strategies  and  switchiTig  to  a new  form  U[’on  detect  ion 
of  failure.  A subt.'lass  of  these  I'roblems  occur  wlu'ii  a robust  ijain 
(one  which  stabiliztjs  each  conf igur.ition  without  regard  for  the 
dynamics  of  structural  changes)  foi-  a set  of  liiK'ar  systems  is 
desired.  The  first  [Jroblem  witliin  this  subclass  deals  with  the 
existence  of  such  a gain.  The  second  ['robli'm  deals  with  tin'  clioice 
of  .m  optimum  robust  gain  with  res|>oct  to  some  cost  indi'x. 

In  the  foll<Jwing  Subsections,  an  exami'le  of  non-switching  gaii\ 
metho<lology  is  <}iven  as  an  i 1 1 ust  rat  i tJn  of  tht'  conce['ts;  since  the 
'lerivations  are  ejuitt'  com['lex,  [iroofs  ar<'  d<'ft'ired  until  I'hai'ter  ■', 
where  the. entire  devi' loi'ment  of  the  non-switching  solufitJti  i.s  (>r<'.':ent  ed  , 


Tho  followiiiy  formulation  is  only  for  the  steady-stato  solution; 
in  Soction  7,  tho  conditions  for  existonce  of  tho  stoady-;itat o solu- 
tion will  bo  given  and  related  to  tho  Uncertainty  Threshold  Principle 
lAthans,et.  al.,  | 


;>.6.1  Problem  Statement 

In  Chapter  5,  the  non-switching  control  problem  is  solved  for 
linear  systems  with  variable  actuator  configurations  and  quadratic 
cost.  It  was  stated  in  the  conclusion  of  the  previous  Section  that 
a relationship  exists  between  the  existence  of  a steady-state  solution 
and  the  Uncertainty  Threshold  Principle.  In  this  Subsection,  the 
existence  of  a steady-state  non-switching  solution  to  the  one  dimen- 
sional example  presented  in  Section  2 will  be  studied  to  illustrate 
this  relationship. 

The  system  to  be  used  is 


^tti=  Vt 


(2.6.1) 


where  x,  a,  b^  and  u are  scalars,  k can  be  either  0 or  1,  and  t takes 
on  integer  values. 


b.  = 
1 


I b if  k=0 


(2.6.2) 


{ lA)  if  k=l 

The  index  k represents  the  structural  state  of  the  system,  and 


is  a random  variable  with  statistics  generated  by  the  Markov  chain 
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(I  = p n , 
-t+1  - - t 


(2.6,3) 


p l-p 
l-l>  p 


(2.6.4) 


where  ii_  ^ ^ is  tlve  probability  tliat  tlie  structural  state  is  i at  time  t, 

qiven  .some  initial  condition  ti  (T  . . ) . 

— init 

The  infinite-time,  or  steady-state  non-switching  control  problem 
is  formulated  by  specifing  that  the  solution  u^  is  to  minimize  tlie 


cost  of  a trajectory  (k  ^ , u ^ ) 


given  by  the  sum 


•’-E 


t=  T 


2 ^ 2 
q X ^ + ru  t 


ini  t 


(2.6.5) 


init 


2.6.2  Summary  of  Solution 

The  solution  is  comi>uted,  from  Chapter  5,  equations  (5.7.17) 


and  (5.7.18),  when  it  exists,  as  the  solution  (S  ,S,)  of 

o 1 


(bSQ+S^/b)bSp 


(bS^+Sj/b)^(r+b^S 
^i(b^S^+Sj^/b^)  tr  4 ('2(b^S^^+Sj/b^)  +r) 


r)^  / 


+ (l-p)  I - 


(bSp+s^/b)S^ 


(bS^+s^/b)  “ (r+Sj^/b^) 
(‘i(b^SQ+Sj/b^)+r)b  4 (‘^(b^S^+Sj/b^) +r)  ^ 


0 r 


(bS^^+S^/b)^(r+b^Sy) 
/b^)+r  4(‘2(b^Sj^tSj/b‘^)+r)^ 

(bS^+Sj/b)^(r+S^/b^) 
4('..(b^Sj^+S^/b^)+r)^ 


(2.6.6) 


+ q 


(2.  6.  7) 
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and  the  control  is  given  by 


(bS^+S^/b)a 

(r+‘i(b^S^+S^/b^)) 


I 


(2.6.8) 


Note  that  the  steady-state  solution  is  a linear  feedbac)c  control 
law  with  a constant  gain  which  is  pre-compvitable  using  equations 
(2.6.6)  and  (2.6.7).  The  on-line  implementation  of  this  solution  has 
the  same  complexity  as  does  the  usual  linear  quadratic  steady-state 
solution. 


1 


i 

1 


i 
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2.7  Existence  of  a Steady-State  Solution  and  the  Uncertainty 

Threshold  Principle . 

In  this  Section,  the  existence  of  a steady-state  solution  to 
equations  (2.6.6)  and  (2.6.7)  is  related  to  the  Uncertainty  Threshold 
Principle  [Atlians  ,37  ] . This  Principle  states  tliat  for  a 

certain  class  of  systems,  there  exists  a threshold,  or  bound,  on  the 
degree  of  uncertainty  in  the  system  dynamics  beyond  which  no  control 
law  will  stabilize  the  system.  Furthermore,  it  is  noted  in 
(Athans  et.  a^.  ,37 ) that  there  does  exist  a "minimizing"  control  even 
though  the  infinite-time  cost  in  infinite. 

For  the  non-switching  gain  class  of  controllers,  it  will  be 
shown  in  this  Section  that,  at  least  for  the  one-dimensional  example 
of  Sections  2 and  6,  such  a threshold  does  exist;  furthermore,  it  will 
be  explicitly  calculated.  In  addition,  it  will  be  demonstrated  that 
the  non-switching  control  gain  converges  even  when  no  finite  cost 
steady-state  solution  exists. 

2.7.1  Formulation  of  Existence  Problem. 

The  question  is  now  asked:  When  does  the  steady  state  solution 
exist?  I.e.,  when  is  the  cost,  given  by 

J = ^2(Sq  + Sj^)Xg  (2.7.1) 

finite? 

This  problem  is  solved  by  showing  when  tlie  solution  does  not 
exist. 

Allowing 

Sq  ••  <»  (2. -7. 2) 


and  setting 


h = lim 


0 , t 


r = lim 


(2.7.3) 


(2.7.4) 


t-f-oo  o,t+l 


where  S and  S are  the  values  of  the  r.h.s.  of  equations  (2.6.61 

L/  f t X ^ Ui 

and  (2.6.7)  iterated  bac)twards  t times  from  an  initial  value  S.  „=Qf 

i , 0 


equations  (2.6.6)  and  (2.6.7)  become 


H'-t 

+ (i-p) 

' h ( 


(b+h/b)b 


(b+h/b) ^b^ 


Js(b^+hA)^)  (b^+hA)^)^ 


_ (b-th/b)  (b+h/b)  ^/b^ 

2 2 ^ 2 22 
»5  (b  +h/b  )b  (b  +h/b  ) 


(2.7.5) 


_ (b+h/b) b (b+h/b) ^b^ 

■*■  **  2 2 2 2 2 
>5  (b  +h/b  ) (b  +h/b  ) 


(b+h/b)  ^ (b+h/b)  ‘ 

>5(b^+h/b^)b  (b^+h/b‘ 


) b^  \ 

b^)^  ) 

?=))) 


(2.7.6) 


2.7.2  Summary  of  Solution. 


Equations  (2.7.5)  and  (2.7.6)  have  5 solutions.  The  solutions  of  h 
and  r of  interest  are: 


For  p *5; 


h = -(p(b'*(6-2W)-3b®-3)  + ( (2b'*-2)p-b^+l)V 

+ (4b®-2b^+2) p^+b®-2b^+l ) / ( (2b^+2) p^-2pW) 


r = a^(-p(b'‘(2p^+4p-2)  + (b“+l)  (p''-2p+l)]’*+(b^+l)p'^) 


*5.  ,^4, 


/((b^+l)^(2p-l) 


(2.7.7) 


(2.7.8) 


V = [b  (p(4-4(b  (2p  +4p-2)+b  (p  -2p+l)+p  -2p+l]  ') 


+2p  - 2 ) +b  ( 5p  - 2p+ 1 ) +p  - 2p+ 1 1 


W = [(b  +2b  +l)p  +(-2b  +4b  -2)p+b  -2b  +11 


4. 
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2.7.3  Graphical  Illustration  of  Solution. 

Equations  (2.7.7)  through  (2.7.12)  are  too  complex  for  much 
information  to  be  gleaned  from  study.  Therefore,  their  significance 


is  demonstrated  graphically  in  this  section. 


These  equations  are  used  to  compute  the  absolute  values  of  a 
versus  b and  p above  which  no  stabilizing  non-switching  control  exists; 
i.e.,  since  T is  the  limiting  ratio  of  S to  S , what  threshold 

value  of  |a|  yields  T = 1?  Since  the  system  (2.6.1)  is  a discrete 
time  one,  this  threshold  quantifies  how  unstable  the  open-loop  system 
must  be  for  there  to  be  no  stabilizing  solution.  This  quantity  is 
called  the  uncertainty  threshold  value  of  |a| . For  the  case  p = *5, 
is  easy  to  compute  from  equation  (2.7.12) 


' threshold 


4 h 
12(b%l)l^ 


(2.7.13) 


For  p / *5, 


I" I threshold  = (b+l)((2p-l) 


4,..  2 


/(P((b  +l)p-fb’(2p  ■»-4p-2)  + (b  +1)  (p  -2p+l]  “)))  ' 

(2.7.14) 

A plot  of  I® I threshold  P (long  axis)  and  b is  shown  in  Figure  2.2. 
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The  ln(b)  axis  is  used  because  a , , , , is  synunetric  witli  respect 

' 'threshold 

to  ln(b)  around  zero  (lal  , . , . (b)  = UL.  , ,,(l/b)  ).  b varies 

' threshold  ' 'tlireshold 

from  e to  e p varies  from  p = 1 to  p = .01  . Note  that 

(a|  , as  b *■  1 and/or  p *■  0.  This  is  because  as  b *■  1 , tlie 

' 'threshold  ^ 

system  looks  more  and  more  like 

= ax^  + bu^  . (2.7.15) 

which  is  controllable  for  all  values  of  a.  As  p »•  0,  the  system  is 

switching  more  and  more  rapidly  between  the  two  structures;  therefore, 

each  structure  has  less  time  to  influence  the  system  unfavorably  and 

the  system  becomes  easier  to  control,  leading  to  lal  ^ 

' 'threshold 


2.7.4  Best  Control  with  Infinite  Cost. 

Although  the  cost  may  be  infinite. 
From  equation  (2.6.8),  and  allowing  S. 


becomes 

* 

u. 


(b-*-h/b)a 

2 2 ^t 

(b  +h/b  ) 


a finite  gain  control  exists. 
-*•  <»  and  ->•  h,  the  control 


(2.7.16) 


Note  that  the  control  gain  does  not  depend  on  q or  r,  but  only  on  p, 
a and  b,  as  in  the  work  with  the  Uncertainty  Threshold  Principle.  A 
plot  of  h versus  p (long  axis)  and  b is  given  in  Figures  2.3a  and  2.3b, 
in  the  same  manner  as  for  T.  Note  that  as  p -*■  0^,  h (except  at 

b = 1) . F’or  this  boundary,  we  rely  on  a symmetric  argument,  switching 
the  roles  of  and  S^,  since  we  only  know  that 

An  interesting  symmetry  exists  in  h with  respect  to  p.  If  h is 
defined  as 


h 


lim  h 
b^O 


(2.7.17) 


N 


Fitjuro  2.^A 


I 


-2.0 


J 


Figure 


L 


-2.5 


■1.0 


-.5 

e 


.3b:  h versus  p,  b. 
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K = ^ 

p 

Lettinq  p = ^2  + x. 


h(x) 


h(x) 


1 - 2x 
1 + 2x 


(2.7.18) 


(2.7.19) 


(2.7.20) 


h (-x) 


Thus,  Inlh(p)]  is  symmetric  around  p = .5  . This  solves  the  boundary 
problem,  because  as  p -*■  1,  h -*•  0 (except  at  b = 1)  , and  the  condition 
Sg  -»■  “>  is  satisfied  (S^^ ->■  0)  . Since  h is  symmetric,  and  h(p,b)  -*-h(p) 
for  p->-0,  the  solution  is  well-defined  at  p = 0. 

In  Figure  2.4,  the  control  gain  divided  by  a,  g,  is  plotted  as  a 


function  of  p and  b. 


(2.7.21) 


Note  that  as  p ^ 0 (and  h -♦  <») , g ->  b,  and  as  p ■>  1 (and  h ->  0)  , 
g -*  1/b,  and  that  b^  = b and  b^^  = 1/b.  Thus,  as  p -»■  0^,  the  optimal  gain 
tends  towards  the  deadbeat  controller  for  the  system  in  structural 
state  1,  and  as  p -*•  1 , the  optimal  gain  tends  towards  the  deadbeat 
controller  for  the  system  in  structural  state  0. 
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2.7.5  Conclusion. 

In  this  Section,  the  steady-stnto  properties  of  the  non-swi tch i luj 

solution  to  a specific  example  of  actuator  failure  were  studioil,  aiul 

were  related  to  the  Uncertainty  Threshold  Principle.  In  particular, 

the  existence  of  an  uncertainty  threshold  has  boon  established,  and 

with  the  help  of  the  high  degree  of  symmetry  in  tlie  example,  the  values 

for  aL.  , given  b and  p,  were  calculated.  It  was  also  shown 

' ' threshold 

that  the  best  control  with  infinite  cost  is  a function  only  of  a,  b and 
p,  a situation  analogous  to  the  solution  obtained  in  tlie  papeis  on  the 
Uncertainty  Threshold  Principle  (Athans  et.  al. , 37) . 

An  analogous  solution  to  that  presented  here  should  exist  for  tlie 
switching  gain  problem,  and  in  fact,  the  rudiments  of  such  a solution 
are  given  in  Section  4.  As  a guide  for  future  research,  it  would  bo 
interesting  to  compare  the  two  methodologies  on  the  basis  of  these 
solutions.  Unfortunately,  it  is  mathematically  intractable  to  extend 
this  result  to  the  multivariable  case,  although  another  approach  may 
be  found. 

2.8  Summary. 

The  unifying  issue  in  this  research  is  the  interrelationship 
between  the  issues  of  control  and  reliability.  Section  7 bruslies  on 
the  question  of  when  a system  design  is  considered  a reliable*  design . 

In  Chapter  3,  a reliable  design  will  be  defined  as  one'  in  which  the 
steady-state  switching  gain  solution  eixists.  The'ri'fore,  e]uestieins 
concerning  the  existence  of  such  solutions  become*  eiuit^e*  impeirtant. 
Unfortunately,  little  hc'adway  has  be'on  maele*  in  tlie  <le»ve' leipme'nt  eif  any 


simple  test  for  the  existe'nce  of  the  steady-state'  solution. 


On 1 y in 
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Section  7,  in  the  specific  case  of  the  non-switching  gain  solution, 
for  a specific  (relatively  trivial)  example,  and  in  Section  4 for  the 
same  example  with  the  optimal  solution,  have  conditions  for  existence 
of  a steady-state  been  resolved.  In  section  7,  these  conditions  are 
given  explicitly;  in  Section  4,  they  are  given  as  the  solution  to  two 
simultaneous  equations.  For  the  general  n-dimensional  problems  in  the 
remainder  of  this  report,  existence  can  only  be  tested  by  iteration  of 
the  solution  equations. 


I 


1 
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CHAPTER  3 

THE  SWITCHING  GAIN  SOLUTION 

3 . 1 Introduction. 

In  this  Chapter,  a control  methodology  for  linear  systems  with 
quadratic  cost  criteria  and  variable  actuator  configurations  will  be 
developed  which  accounts  for  the  failure,  repair  and  reconfiguration 
of  the  actuators  by  switching  the  control  gain  on  detection  of  a 
change  in  configuration.  This  problem  is  viewed  as  a control  problem 
rather  than  as  the  traditional  estimation  problem.  Therefore,  a 
deterministic  model  is  assumed,  except  for  the  random  clianges  in 
configuration,  which  are  modeled  by  a Markov  chain.  This  metliodology 
has  the  advantage  that  all  gain  and  expected  cost  calculations  are 
done  off-line.  The  gains  switch  on-1  ine  with  changes  in  the  configura- 
tion, which  are  observable  with  one-step  delay  for  almost  all  values 
of  u^  (i.e.,  except  for  a set  of  measure  zero).  In  addition,  the 
method  is  useful  in  the  stochastic  case,  tliougli  not  optimal,  in 
conjunction  with  identification  methods  such  as  liypothesis  testing 
and  dual  identification,  which  will  be  described  in  Chapter  4.  Tlie 
gain  and  expected  cost  calculations  can  be  used  as  an  evaluation 
technique  in  computer-aided  design  of  linear  systems.  An  example 
would  be  in  trade-off  studies  of  various  redundancy  configurations 
witfi  respect  to  performance,  reliability,  and  system  effectiveness. 

The  disadvantages  of  the  technique  as  it  is  presented  here  are  that  it 
requires  perfect  measurement  of  the  state  and  that  only  multiple 
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actuator  configurations  are  considered.  The  multiple  sensor  configura- 
tion problem  should  bo  dual  to  this  work.  Changes  in  the  A matrix 
are  a minor  extension;  however,  tl\e  general  problem  allowing  variations 
in  both  the  actuators  and  the  observers  would  be  a major  result. 

Previously,  several  authors  have  studied  the  optimal  control  of 
systems  with  randomly  varying  structure.  Most  notable  among  these  is 
[Wonham,221,  where  he  develops  a solution  to  the  linear  regulator 
problem  with  randomly  jumping  parcuneters  in  continuous  time.  The 
solution  assumes  apriori  that  the  controller  has  perfect  information 
about  the  present  state  of  the  random  parameter  process.  Little  work 
was  done  on  the  steady-state  existence  problem. 

The  solution  presented  in  this  Chapter  is  analogous  to  that  of 
Wonhcun's;  however,  the  discrete  time  formulation  of  the  problem  allows 
the  controller  to  observe  exactly  with  one  step  delay  the  value  of  the 
Markov  parameter  process.  Thus,  it  is  shown  that  for  the  discrete- 
time process,  the  optimal  controller  is  not  dual.  ' 

In  addition  to  this  conclusion,  this  research  makes  the  connection, 
for  the  first  time,  of  control  and  system  reliability  and  effectiveness. 
This  is  the  unifying  concept  in  the  entire  report,  and  has  been  discuss- 
ed in  detail  in  Chapter  1. 

The  procedure  for  determining  the  existence  of  a steady-state 
solution  to  the  switching  gain  control  problem  divides  system  designs 
into  two  classes:  If  a design  allows  a steady-state  solution,  then 
that  solution  is  stabilizing  (see  Section  7,  Chapter  5);  therefore, 
that  design  is  classified  as  a reliable  design.  On  the  other  hand,  if 
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no  steady-state  solution  exists,  then  that  design  is  classified  as 
inherently  unreliable. 

Although  no  easy  test  exists  for  the  existence  of  a steady-state 
solution,  the  computer  can  always  be  used  to  iterate  equation  (3.3.6) 
backward  in  time  and  check  for  stability.  Therefore,  this  methodology 
yields  a classification  of  systems  into  those  which  are  inherently 
reliable  and  those  which  are  not. 


3.2  Mathematical  Formulation. 


In  tills  S«>ction,  the  n-dimensional  extension  to  the  one-dimension- 
al switching  gain  result  presented  in  Chapter  2 will  be  developed. 

The  only  non-trivial  task  is  to  prove  that  the  system  structure  is 
observable  for  almost  all  values  of  the  control.  The  system  model  is 


5t.l*  * ^^k(t)?it 


where 


X ^ e R 


u ^ e R 


. ..n  n 

A e R 

and,  for  each  k,  an  element  of  an  indexing  set  I 

k C I = {0,1,2,  . . . ,l} 

„ „n  X m 
B , e R 
— k 

where 


B ^ e (b  . } . . 

— k - 1 1 e I 


(3.2.1) 

(3.2.2) 

(3.2.3) 

(3.2.4) 

(3.2.5) 

(3.2.6) 


(3.2.7) 

The  index  k(t)  is  a random  variable  taking  values  in  I which  is 


governed  by  a Markov  chain  and 
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IT  , = P ■’’f  ^ 

-t+1  1 


II  e R 
- 1 


L+1 


(3.2.8) 

(3.2.9) 


wliere  H.  is  the  probability  of  )c{t)  = i,  given  no  on~l ine  information 
i / 1 

about  )c(t),  and  ii^  is  the  initial  distribution  over  I. 

It  is  assumed  that  the  following  sequence  of  events  occurs  at 
each  time  t: 

1)  observed  exactly 

2)  switches  to 

3)  then  u^  is  applied. 


The  control  interval  is  assumed  to  be 

{0,1,2,  . . . ,t} 

and  the  cost  function  is  selected  as 
T-1 

Jt  < <2it'-t^t=0  '-T  ^ 


(3.2.10) 


= 2 


T T T ^ 

x,.Qx^  + u.  Ru.  + X Qx 
— t*-— t — t 1 —Ti.  — T 


(3.2.11) 


t=U 

The  objective  is  to  choose  a feedbac)t  control  law,  which  may 

into  u 

(3.2.12) 

(3.2.13) 


depend  on  any  past  information  about  , mapping  x^  


A.*  r,” 

it  ■ ^ 


•^t 


^t  = ^t 

such  that  the  expected  value  of  the  cost  function  from  equation 

(3.2.11) 


is  minimized  over  all  possible  mappings  i ^ at 


(3.2.14) 
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3 . 3 The  Switching  Gain  Solution. 

Normally,  a control  law  of  the  form  (3.2.13)  must  provide  both 
a control  and  an  estimation  function  in  this  type  of  problem;  hence 
the  label  dual  control  is  used.  Here,  the  structure  of  the  problem 
allows  the  exact  determination  of  k(t-l)  from  ^ and 

for  almost  all  values  of  u ^ . This  result  is  stated  and  proved  in 

the  following  theorem. 

Theorem  1:  For  the  set  {b,  } , ,,  where  the  B,  's  are  distinct,  the 

— k k G I — k 

set  {x,  . = Ax  +B,u^}^'„  has  distinct  members  for  almost  all 

— k,t+l  1 — k — t k=0 

values  of  u^  . 

Proof ; See  Appendix  3.1. 


Ignoring  the  set  of  controls  of  measure  zero  for  which  the 
members  of 


{x. 


(3.3.1) 


-k,t+l  k=0 

are  not  distinct,  then  for  (almost)  any  control  which  the  optimal 

algorithm  selects,  the  resulting  state  x^^2.  compared  with  the 

members  of  the  set  (3.3.1)  for  an  exact  match  (of  which  there  is  only 

one  with  probability  1),  and  k(t)  is  identified  as  the  generator  of 

that  matching  member  x , ^ . 

— k,t+l 

Since  perfect  identification  is  the  best  any  algorithm  can  actiieve, 

* * 


the  optimal  control  law  u = 4>..(x^)  can  be  calculated  with  the 

— t t — t 

assumption  that  k(t-l)  is  known,  since  this  is  tlie  case  with  probability 
one.  Thus,  this  solution  will  be  labeled  the  switching  gain  solution, 
since,  for  each  time  t,  L+1  optimal  solutions  are  calculated  apr iori , 


and  one  solution  is  chosen  on-line  for  each  time  t,  based  on  the  past 
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measurements  ^ and  , which  yield  perfect  knowledge  of 

k(t-l). 

Dynamic  programming  will  be  used  to  derive  the  optimal  switching 
gain  solution.  At  each  time  t,  the  expected  cost-to-go  using  the 


control  sequence 


-t  ' -t4l  ' -t+2  ' • • • ' 


-T-1 


(3.3.2) 


and  given  the  value  of  k(t-l)  is  defined  as 


V(x  ^ ,u  ^ ,k(t-l)  ,t) 


x^2x,  + u.'Ru 


t - 1 1 

* 


^k(t)^''  (x^^j  ,k(t)  ,t+l)  |k(t-l)} 


(3.3.3) 


where  * denotes  the  optimum  value  and  V*  is  the  optimal  value  of  V. 


Then,  by  dyneimic  programming 


V (x^  ,k(t-l),t)  = min 


iit= 


(T  T 

Xt2^t  * iit^^t 


+ E 


It  is  proved,  from  Appendix  3.2,  that 
V (x^  ,k(t-l),t)  = k t-t 


k(t){''*‘-t+l  '’^<t),t+l)  I k(t-l)[j  (3.3.4) 


(3.3.5) 


where  the  £ are  determined  by  a set  of  L+1  coupled  Riccati-like 

K f t 

equations  (one  for  each  possible  configuration) : 


Sk.t  - ' 


^ik  -i,t+l 


-IS 

■IS 


P . . S , ^ , B . 
*^ik  — i,t+l  —1 


^ik  -i-i,t+l 


R + 


p..  B . s . B . 
ik  —1  — i,t+l  — 1 


-1 


!a. 


A + 2 


(3.3.6) 


I 


I 


The  optimal  control,  given  k(t-l)  = k,  is 

u*.  . = - [r  + ^ P-v  B^S. 

- k,t  I—  “"^ik  — 1 — 1,  t+1  — 1 


-1 


Writing 
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p..  B . S . . Ax 

^ik  —1  — i.,t+l t 


(3.3.7) 


— k,t  — k,t— t 


then 


^k,t 


R + > , P.,  B . S . B . 

— A-i  ik  — 1 — i,t+l  — 1 


-1 


(3.3.8) 


• > p.,  B . S . ^ , A 

ik  — 1— i,t+l  — 


(3.3.9) 


Thus,  H ^ ^ (x  ^.)  is  a switching  gain  linear  control  law  which 

depends  on  k(t-l).  The  variable  k(t~l)  is  determined  by 

k(t--l)  = i iff  2i^=  Ax  ^ 2+  (3.3.10) 

Note  that  the  S . ' s and  the  optimal  gains  G can  be  computed 

1 , t k , t 

off-line  and  stored.  Then,  at  each  time  t,  the  proper  gain  is  selected 
on-line  from  k(t-l),  using  equation  (3.3.10),  as  in  Figure  3.1. 


3,4  Discussion  of  Results. 

The  solution  in  section  3 is  quite  complex  relative  to  the  struc- 
ture of  the  usual  linear  quadratic  solution.  Each  of  the  Riccati-like 
equations  (3.3.6)  involves  the  same  complexity  as  the  Riccati  equation 
for  the  linear  quadratic  solution.  In  addition,  there  is  the  on- lint' 
complexity  arising  from  the  implementation  of  gain  scheduling.  In 
Chapter  5,  a non-switching  gain  solution  will  be  presented  which  has 


i! 


J 


an  identical  on-line  structure  to  that  of  the  linear  quadratic 
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solution,  but  has  similar  of f-line computational  complexity  to  that  of 
the  switching  gain  solution.  Depending  on  the  system  requirements, 
either  solution  could  be  used;  the  non-switching  gain  solution  is 
suboptimal,  but  requires  less  on-line  complexity.  This  trade-off  may 
favor  the  non-switching  solution  in  some  cases. 

A steady-state  solution  to  equation  (3.3.6)  may  exist,  but  the 
conditions  for  its  existence  are  unknown.  The  steady-state  solution 
would  have  the  advantage  that  a time-invariant  set  of  gains  result. 
Thus,  only  one  set  of  gains  need  be  stored  on-line,  instead  of  requir- 
ing a set  of  gains  to  be  stored  for  each  time  t.  Since  the  steady- 
state  solution  is  simply  the  value  to  which  equation  (3.3.6)  converges 
as  it  is  iterated  backward  in  time,  at  present,  the  equations  can 
be  iterated  numerically  until  either  they  converge  or  meet  some  test 
of  non-convergence.  Unlike  the  non-switching  solution  presented  in 
Chapter  5,  the  possibility  of  limit  cycle  solutions  in  the  switching 
gain  computations  is  excluded  by  the  following  lemma: 

Lemma  1 ; If  the  optimal  expected  cost-to-go  at  time  t is  bounded 
for  all  t,  then  equation  (3.3.6)  converges. 

Proof ; See  Appendix  3.3. 

Once  again,  it  is  stressed  that  the  existence  of  a steady-state 
solution  to  the  switching  gain  problem  establishes  a division  of 
system  designs  into  those  which  are  inherently  reliable  and  those 
which  are  unreliable.  Even  though  conditicns  to  test  for  the  exis- 
tence of  the  steady-state  solution  are  unavailable , software  can  be 
used  with  iteration  for  the  test. 
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3. 5 Examples. 

In  this  Section,  a two-dimensional  example  is  presented  with  three 
different  switching  gain  solutions  to  illustrate  the  switching  gain 
computational  methodology.  The  computer  routines  which  are  used  in 
the  calculation  of  the  switching  gain  solution  are  listed  in  the 
Appendix.  The  primary  subroutine  is  READY;  it  calls  WEIGHT.  Any  other 
routines  which  are  used  are  from  the  standard  ESL  subroutine  library. 

The  main  program  RDYMAIN  is  used  to  call  READY. 

Example  3.1  is  a two-dimensional  system  with  four  structural 
states  corresponding  to  the  failure  modes  of  two  actuators.  In  this 
example,  failure  of  an  actuator  is  modeled  as  an  actuator  gain  of 
zero.  Thus,  the  four  structures  are:  I)  Both  actuators  working  (B^  ); 
II)  One  actuator  failed  and  ),  and  III)  Both  actuators  failed 
(B^  ).  The  system  is  controllable  in  all  structures  except  for  the 
sturcture  represented  by  B ^ . 

Actuator  failures  and  repairs  are  assumed  to  be  independent  events 
with  probabilities  of  failure  and  repair,  per  unit  time,  of  p^  and  p^, 
respectively,  for  both  actuators. 

In  Example  3.1,  the  matrixes  £ and  R are  the  quadratic  weighting 
matrices  for  the  state  x^  and  the  control  , respectively.  The 
matrix  is  the  Markov  transition  matrix,  which  is  calculated  from  knowl- 
edge of  the  system  configuration  dynamics,  represented  graphically 


in  Figure  3.2. 


r 


\ 


There  are  three  Cases  to  Example  3.1.  Each  Case  assumes  a different 
failure  rate  and  repair  rate  for  the  actuators.  Case  i)  has  a high 
probability  of  failure  and  a low  probability  of  repair,  relative  to 
Cases  ii)  and  iii) . The  switching  gain  solution  is  not  convergent  for 
Case  i) ; the  gains  themselves  converge,  but  the  expected  costs  do  not. 
Only  configuration  state  0 is  stabilized  with  its  corresponding  gain. 


Cases  ii)  and  iii)  both  assume  more  reliable  actuators  than  does 
Case  i) . Both  Cases  ii)  and  iii)  have  convergent  switching  gain 
solutions.  Therefore,  both  Cases  ii)  and  iii)  represent  reliable 
configuration  designs , while  Case  i)  is  unreliable.  This  difference 
is  due  entirely  to  the  different  component  reliabilities.  Equivalently, 
Cases  ii)  and  iii)  are  stabilized  by  the  switching  gain  solution,  while 
Case  i)  is  not.  Note  that  in  this  Example,  stabilizability  is  not 
equivalent  to  stability  in  each  configuration  state,  or  robustness. 

For  this  example,  no  robust  gain  exists  because  the  system  is 
uncontrollable  from  configuration  state  3. 

Cases  ii)  and  iii)  are  also  presented  in  Chapter  5,  where  their 
non-switching  gain  solutions  are  given.  According  to  the  theory,  it 
should  be  more  difficult  to  stabilize  a given  system  with  the  non-switch- 
ing gain  than  it  is  with  the  switching  gain,  because  of  the  optimality 
of  the  switching  gain  solution.  This  is  demonstrated  for  this  example; 
in  Chapter  5,  the  non-switching  gain  solution  to  Case  ii)  is  not 
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convergent. 
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Example  3.1,  Case  ii) 
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3.6  Summary. 

In  this  chapter,  the  optimal  solution  to  the  linear  control 
problem  with  variable  actuator  configuration  was  developed.  It  was 
shown  that  the  optimal  solution  uses  a linear  switching  feedback  gain 
which  depends  on  the  previous  configuration.  This  configuration  is 
directly  computable  from  the  past  measurements;  this  fact  allows  the 


development  of  the  switching  gain  solution  by  eliminating  dual  con- 
trol considerations.  The  exact  measurement  of  the  configuration  with 
one-step  delay  holds  only  for  the  deterministic  case,  where  there  is 
no  corruption  of  the  state  or  control  observations  by  noise. 

In  Chapter  4,  the  use  of  the  switching  gain  methods  will  be 
demonstrated  for  stochastic  problems  in  conjunction  with  two  different 
forms  of  identification;  Hypothesis  testing  and  dual  identification, 
a technique  for  "pushing"  the  control  variable  out  of  the  noisy 
region,  when  the  noise  is  amplitude  limited,  to  obtain  an  exact 
identification  of  the  system  structure. 


r 

I 


! 


I 


It 
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CHAPTER  4 


EXTENSIONS  TO  THE  STOCHASTIC  CASE 

4.1  Introduction . 

In  Chapter  3,  the  optimal  solution  to  the  deterministic  linear 
quadratic  control  problem  with  variable  actuator  configuration  was 
developed.  It  was  also  demonstrated  that  the  optimal  solution  of 
the  general  stochastic  linear  quadratic  problem  is  hopelessly  complex 
in  Chapter  2.  Therefore,  in  this  Chapter,  extensions  to  the  deter- 
ministic solution  to  allow  its  operation  in  a stochastic  ^ nvironment 
will  be  studied. 

From  the  derivation  of  the  switching  gain  solution,  whenever 
the  structure  of  the  system  is  known  perfectly  with  one  step  delay, 
and  if  it  is  assumed  that  it  will  be  measured  perfectly  at  the  next 
time  instant,  the  optimal  solution  is  the  deterministic  switching 
gain  solution.  In  designing  a suboptimal  control  system,  a method 
of  identifying  the  system  structure  is  used,  with  the  assumption  that 
the  identification  is  perfect,  and  the  appropriate  deterministic 
gain  is  selected. 

Two  conceptually  different  methods  of  structure  identification 
will  be  presented  in  this  Chapter.  The  first  is  classical  hypothesis 
testing.  It  is  the  easiest  to  implement,  although  extensions  to 
n-step  hypothesis  testing  can  be  made  which  are  very  complex.  The 
second  method  is  labeled  dual  identification;  the  expression  is  used 
because  it  takes  advantage  of  the  dual  effect  of  the  control  law  to 
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guarantee  perfect  identification.  In  this  method,  a perturbation 


(which  may  or  may  not  be  that  small)  to  the  deterministic  control  is 


introduced  which  separates  the  effect  of  amplitude  limited  white 


control  noise  from  that  of  the  system  structure.  As  a worst  case 


control  law,  this  perturbation  would  be  applied  at  each  time  instant. 


but  in  practice,  it  would  only  be  applied  once  every  n time  instances 


so  that  its  overall  effect  on  system  performance  would  be  lessened. 


In  the  next  Section,  the  system  mod  1 will  be  described, and  the 


hyp)othesis  testing  identification  algorithm  will  be  presented. 


Testing  Identification. 


The  system  model  used  here  is  the  same  as  in  Chapter  3,  but  with 


the  exception  that  additive  white  noise  is  introduced  into  the 


dyncimics : 


(4.2.1) 


For  the  hypothesis  testing  identification  method,  is  assumed  to  be 

zero  mean  white  noise  with  probability  distribution  p(£).  It  is 


assumed  to  be  uncorrelated  with  k(t)  and  x^  . Perfect  measurement  of 


the  state  is  retained. 


The  basic  hypothesis  testing  method  is  very  simple:  At  each  time 


t,  one  of  L+1  hypotheses  is  chosen,  where  each  hypothesis  is 


H.  : k(t-l)  = i 
1 


(4.2.2) 


With  each  hypothesis  H^,  there  is  a probability  of  being 


correct,  given  the  measurement  x^  and  the  past  information:  TT(t-l  1 1-1)  , 


the  probability  distribution  of  k(t-l),  given  the  measurements  through 


— 1-1  * Then  the  updated  probability  (see  App_.idix  2)  7i^(t-l|t),  the 


probability  of  k(t-l)  = i,  given  all  measurements  through  , is 


given  by 
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Tr.(t-l|t)  = 


P(^t  - ^^t-1  - 


--t-1  ” 5- j H-t-i 


(4.2.3) 


Hypothesis  is  assumed  to  be  correct  if 
iT^(t-llt)  > iTj(t-l|t)  for  all  j / i 


(4.2.4) 


Ties  are  resolved  arbitrarily.  Then,  given  the  correct  hypothesis  H^, 
the  corresponding  deterministic  optimal  switching  gain  is  used  to 
compute  the  control  at  time  t 


u . = G . ^ X ^ 

— t — i,t— t 

as  in  equations  (3.3.8)  and  (3.3.9). 


(4.2.5) 


The  probability  distribution  is  then  propagated  with  the  Mar)cov 
chain  equation 


Tr(t|t)  = Pn  (t-l|t) 


(4.2.6) 


and  the  process  repeats. 

This  algorithm  can  worlc  well  if  there  are  significant  differences 
in  the  effect  of  the  control  variable  between  configurations.  When 
the  differences  are  slight,  a mistraclcing  will  result  until  the  errors 
are  large  enough  to  be  detected  through  equation  (4.2.3).  The  method 
does  not  exploit  any  of  the  dual  effect  of  the  control  variable  on 
the  measurement  of  the  configuration.  The  method  presented  next  does 
use  the  dual  effect  to  identify  the  correct  structure.  Analytically, 
it  cannot  be  said  which  method  is  best,  as  the  optimal  control  law 
will  lie  somewhere  between  the  two.  It  is  possible  to  extend  the 
hypothesis  testing  procedure  to  n-step  hypothesis  testing  where  a 


hypothesis  is  made  about  the  last  n values  of  k(t)  and  is  then  tested. 


P I 


B 


since  this  investigation  is  not  within  the  primary  scope  of  this 
research,  it  is  left  as  an  open  problem  for  future  research.  It  is 
also  possible  that  a combination  of  hypothesis  testing  and  dual  identi- 
fication may  be  used  to  gain  some  of  the  advantages  of  both  methods; 
dual  identification  yields  fast  identification  of  the  correct  structure, 
while  hypothesis  testing  does  not  sacrifice  control  of  the  system 
while  there  is  a high  probability  that  the  structure  is  correctly 
identified. 


4.3  Dual  Identification. 


The  underlying  concept  of  dual  identification  is  to  periodically 
change  the  control  in  order  to  increase  the  accuracy  of  identification 
of  the  structure.  In  the  limiting  case,  the  control  is  changed 
enough  to  guarantee  perfect  identification  of  the  current  structure 
with  the  next  observation.  For  this  case  only  amplitude  limited  noise 
is  considered.  The  system  model  is 

(4.3.1) 


-t+1  --t  -)c(t)  -t  -^t 


where  is  S,-dimensional  white  noise  which  takes  on  values  in  the 


unit  sphere  with  distribution  p (^)  and  is  uncorrelated  with  x^  and 


k(t).  M is  an  n x £ matrix  which  defines  the  ellipsoid  in  k which 
contains 

Normally,  if  no  identification  were  to  be  performed,  and  if  k(t-l) 


were  known,  the  optimal  deterministic  switching  gain  G.  from 

K V t**  X ) / u 


equation  (3.3.9)  would  be  used  to  compute  u*  . 


-t  -k(t-l),t-t 


(4.3.2) 

In  dual  identification,  the  goal  is  to  compute  a gain  offset  u 

A # t 


'i 
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such  that  when  the  control 


(4.3.3) 


is  applied  to  the  system,  identification  of  the  structure  k(t)  with 
the  observation  is  guaranteed.  To  accomplish  this,  note  that, 

for  a given  will  be  in  a bounded  convex  set  determined  by 


B,  and  M.  Thus, 
— k — 


^t+1 


-Ax.  =B.u.  +M5. 

t — k— t — ^t 


(4.3.4) 


and  can  be  any  element  in  the  unit  sphere  S(R  ).  Therefore, 
perfect  identification  of  k(t)  is  guaranteed  if  no  two  of  the  domains 
of  corresponding  of  the  Bj^'s  have  a non-empty  open  intersection. 

That  is,  the  following  condition  must  be  satisfied  for  each  pair  of 

j, 

Bj^  's  and  every  and  ^2  ): 

i 


-B^^lU^  .M({j  -5^1  ^ 0 


(4.3.51 


This  condition  is  the  saune  as 


M (iu  - II 

1 2 


if  <5.u  ~ 5.V  ^ n(m) 

*"1  '^2 


Otherwise , 


(Bfe  - B.  )u  0 

Kf  x^ 


(4.3.6) 


where  m"  is  the  generalized  inverse  of  M and  N(M)  is  the  nullspace 
of  M.  Note  that  the  inequality  of  (4.3.6)  can  be  relaxed  to  equality, 
since  the  intersection  of  the  two  domains  of  would  only  be  at 

the  point  of  tangency,  a set  of  measure  zero  in  either  domain. 

The  objective  is  to  choose  u such  that  (4.3.6)  is  satisfied 
for  all  pairs  B and  B in  the  reachable  subset  of  all  actuator 
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configurations.  The  reachable  subset  refers  to  the  subset  of  configu- 
rations which  have  a non-zero  probability  of  occurance  at  time  t, 
given  that  the  configuration  was  ®]^(t-l)  '^^is  is  the  same  as 

the  condition  that 


B . is  in  the  reachable  subset  from  B , , 

-i  -k(t-l) 

Pik(t-l) 


(4.3.7) 


Suppose  that  there  are  J configurations  in  the  reachable  subset  from 
. Then  there  are  J(j+l)/2  pairs  of  configurations  for  which 
condition  (4.3.6)  must  be  satisfied.  Also,  since  u affects  the 

1ft 

state  reasonable  to  minimize  its  effect.  Therefore, 

since  the  effect  of  u is  modified  by  B , . , it  is  reasonable  to 

minimize  the  norm  of  . Thus,  the  minimization  problem  is  formu- 

lated  subject  to  the  constraints  (4.3.6). 

Hi  t II  ^ 

^Ift  ' 


subject  to 

4 - II  [u%  < 0 


(4.3.8) 


where 


,=  M (B.  - B.) 

— i+(]-l)*J  — —1  —3 


(4.3.9) 


Formulating  this  as  a nonlinear  programming  problem,  the 


Hamiltonian  is 


. II 


* 5 


> 0 

\ ° if  ■‘-llEfc  lit  i ill, t' II'' 


(4.3.10) 


(4.3.11) 


Differentiating  H with  respect  to  \,  and  solving  for  u as  a 

1 1 1 


function  of  u^  and  the  parameter 
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3u 


= 0 = 2 


l,t 


u,  . - y]  2X,  D D,  [u^  + u , ^ ] (4.3.12) 


'k-k-k^-t  ■ -l,t 


or, 


U,  „ = [I  - y.A.  d!^  D.-  u ! 

— l,t  — k — k — k k — k — k — t 


(4.3.13) 


Now,  using  (4.3.13)  in  the  constraint  equation  (4.3.11) 

4 - 11  D [I  + [I  - 5Z\^k^k  ’-t  ° 


Noting  that 

[I.  - \U  + Ad’’d]"^d’’ D]  = [1  + Ad^d]"^ 

then  (4.3.14)  simplifies  to 

4 - II  Ek  " 2 V^k^k  ^'^Jit  11  - ° 


(4.3.14) 


(4.3.15) 


(4.3.16) 


and  if  (4.3.16)  is  a strict  inequality,  then  = 0.  In  general, 
a numerical  algorithm  must  be  used  to  solve  for  ^ in  the  set  of 
equations  (4.3.16);  this  can  be  a major  drawback  to  the  application 
of  this  methodology  if  the  on-line  computer  resources  are  unavailable. 
Although  the  computational  burden  of  this  technique  is  a disadvantage, 
dual  identification  would  most  likely  be  implemented  in  combination 
with  a hypothesis  testing  algorithm.  Dual  identification  would  then 
form  a test  to  be  performed  on  the  system  after  some  interval  of  time 
to  ensure  that  the  hypothesis  testing  algorithm  correctly  tracked  the 
configuration. 
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4.4  Examples. 

In  this  Section,  the  one-dimensional  example  of  Chapter  2,  Section  2 
in  implemented  with  additive  white  noise  applied  to  the  control  input. 
Three  suboptimal  control  algorithms  derived  from  this  Chapter  are  imple- 
mented: Hypothesis  testing,  dual  identification,  and  hypothesis 

testing  in  combination  with  dual  identification,  which  is  utilized  every 
fifth  time  instant.  The  purpose  of  this  example  is  to  Illustrate  the 
degrading  effect  of  the  dual  identification  algoritlun  on  the  system 
state. 

The  principle  subroutine  used  to  generate  the  computer  simulations 
of  Example  4.1  is  SWITCH;  it  is  listed  in  the  Appendix.  SWITCH  calls 
FIG  and  UCALC,  also  in  the  Appendix;  any  other  routines  which  are  used 
are  in  the  ESL  subroutine  library. 

The  system  in  Example  4.1  has  two  structures,  represented  by  the 
matrices  (b  = 2.)  and  B (1/h  •=  .5);  the  Markov  transition  probabili- 

ties are  given  by  the  matrix  P.  The  switching  gain  solution  was  calcu- 
lated using  the  software  described  in  chapter  3,  Section  5.  Case  i) 
of  the  Exeunple  corresponds  to  the  hypothesis  testing  methodology  described 
in  Section  2.  The  additive  white  noise  was  amplitude- limited  with  zero 
mean  and  variance  5=1.  Case  ii)  of  the  example  demonstrates  the  perfor- 
mance degradation  due  to  the  exclusive  use  of  dual  identification.  Note 
that  the  variation  among  the  values  of  the  state  and  control  are  larger 
than  in  Case  i).  The  advantage  of  dual  identification  is  that,  for 
amplitude-limited  white  noise,  perfect  identification  of  the  system 
structure  with  one-step  delay  is  guaranteed.  In  Case  iii),  hypothesis 
testing  is  used  four-fifths  of  the  time  to  partially  avoid  the  degradation 
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due  to  dual  identification.  The  control  is  more  effective  in  Case  iii) 
than  in  Case  ii);  however,  for  this  example,  it  is  not  clear  that  the 
use  of  dual  identification  one-fifth  of  the  time  is  warranted,  since  a 
performance  degradation  of  Case  iii)  over  Case  i)  is  still  evident  in 
this  particular  simulation.  More  simulation  would  have  to  be  carried 
out  before  the  proper  ratio  of  the  use  of  hypothesis  testing  to  the 
use  of  dual  identification  could  be  determined. 


Example  4 . 1 : 


1.414 


Bq  = 2.000 


= .5000 


P = 


Q =»  3.000  R = 1.000 

‘’.7  .3 

.3  .7 

Switching  Gain  Deterministic  Solution; 

Gq  = -.7569 

Gj  = -1.008 

The  system  dyn^unics  are 

t+1  t k(t)  t 
k(t)  G {0,1} 

The  cost  function  which  was  minimized  is 


t=0 


Q\  + 


Ru^  I 1 


where 

TT  = [»S  *5]’’ 

Structural  transitions  are  of  the  form 


.3 


When  dual  identification  was  employed,  the  control  was  set  to 

* 

u^  = 1. 25 (sign (u^) ) 

This  control  was  the  minimum  value  required  to  establish  perfect 


identification 
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4.5  Sunanary. 

In  this  Chapter,  two  methods  have  been  proposed  to  extend  the 
deterministic  optimal  switching  gain  solution  of  Chapter  3 to  the 
stochastic  case.  The  two  methods  represent  the  two  fundamental 
concepts  of  identification:  Estimation  and  dual  control.  The 
optimal  stochastic  control  law,  if  it  could  be  computed,  would  rely 
on  both  concepts,  using  estimation  when  the  control  variable  is 
large  (and  the  state  is  far  from  the  origin)  and  dual  control  to 
enhance  estimation  when  the  control  and  state  variables  are  small. 

In  the  dual  identification  technique  presented  here,  control  is 
sacrificed  to  obtain  an  exact  observation  of  the  structure.  Thus, 
the  system  response  would  be  roughly  periodic,  with  the  state  being 
driven  away  from  the  origin  in  order  to  obtain  an  accurate  estimate 
of  the  configuration,  and  decaying  back  toward  zero  between  identifi- 
cations. In  the  period  when  the  control  is  not  modified,  hypothesis 
testing  would  be  used  to  track  the  configuration. 


CHAPTER  5 


THE  NON-SWITCHING  GAIN  SOLUTION 


5. 1 Introduction 

In  the  previous  two  chapters,  the  switching  gain  solution  was 
developed  and  studied.  In  this  chapter,  attention  will  be  focused 
on  obtaining  a constant,  robust,  or  non-switching  gain  which  solves 
a variable  actuator  configuration  linear  quadratic  control  problem, 
with  minimum  cost  for  this  class  of  solutions.  It  must  be  stressed 
that  this  is  a suboptimal  solution;  for  the  deterministic  case. 

Chapter  3 gives  the  optimal  solution.  The  interest  in  this  chapter 
lies  in  determining  a sequence  of  gains,  for  a linear  control  law, 
which  do  not  switch  in  response  to  the  detection  of  a change  in  system 
structure.  For  instance,  it  may  be  desirable  to  ensure  the  stability 
of  a control  system  under  certain  types  of  failure  without  creating 
the  complexity  necessary  to  detect  those  failures  and  compensate  for 
them,  as  is  done  in  the  switching  gain  solution. 

This  class  of  solutions  is  related  to  the  overall  robustness 
problem  where  fault-tolerant  control  systems  are  desired.  Although 
not  formulated  in  this  manner,  the  research  described  in  this  Chapter, 
as  in  Chapter  3,  is  readily  extendable  to  system  with  variable  system 
matrices  as  well;  i.e.,  where  the  system  can  be  represented  as  a set 
of  possible  structures  over  some  suitable  index,  even  though 

this  class  of  problems  is  not  as  directly  related  to  the  underlying 


I- 


i 

I 

102 

reliability  theme  of  this  report. 

Non-switchinq  >.jain  solutions  to  the  variable  actuator  configura- 
tion class  of  problems  can  be  obtained  in  different  mathematical  ways.  ; 

Problem  A of  Section  J is  reformulated  as  a deterministic  control 
problt'm  (Problem  AE) , and  is  solved  using  the  necessary  conditions  of 
the  Matrix  Minimum  Principle  (Athans,41]  in  Section  5.  Unfortunately 

this  approach,  although  yielding  the  necessary  conditions  for  an  opti-  ’ 

mum,  does  not  allow  an  analytic  solution.  Therefore,  in  Section  6, 
a second  problem  (Problem  B)  is  formulated  and  solved  using  dynamic 
programming . 

Section  7 is  by  far  the  most  detailed  and  one  of  the  most  imfHDr- 
tant  sections  of  the  report,  along  with  Sections  8 and  9.  In  Section 
7,  the  concepts  of  stability  and  cost-stability  are  defined  and  are 
used  to  prove  an  equivalence  between  the  infinite-time  versions  of 
Problems  AE  and  B.  In  Subsection  7.6,  the  steady-state  solutions  for 
both  problems  are  defined.  Unfortunately,  nothing  in  the  mathematics 
appears  to  rule  out  the  possibility  of  limit  cycles  in  the  infinite- 
time solution;  this  is  discussed  in  Subsection  7.7.  When  the  constant 
steady-state  solutions  to  the  two  problems  exist,  it  is  proved  in 
Section  8 that  they  are  identical.  This  is  a very  important  result,  as 
it  allows  the  steaey-state  solution  of  a complex  two-point  boundary 
value  problem  which  is  much  more  tractable. 

In  Section  9,  it  is  demonstrated  that  the  general  robustness  problem 


for  linear  systems  (where  one  wishes  to  determine  a single  stabilizing 
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gain  for  a set  of  linear  systems)  is  solved  in  this  framework  for  the 
class  of  systems  with  variable  actuator  configurations.  Examples  of 
both  the  non-switching  solution  to  Problem  B and  the  robustness 
result  are  given  in  Section  10,  and  a chapter  summary  in  Section  11. 

5 . 2 Problem  Statement. 

The  objective  of  the  research  described  in  tliis  Chapter  is  to 
form  a methodology  which  will  be  used  to  compute  apriori  a gain  ^ 

(either  time-varying  or  steady-state)  which  minimizes  the  expectation 
of  the  quadratic  performance  index  over  a set  of  linear  systems  with 
actuator  variation  and  known  transition  probabilities  of  structural 
change  (Problem  A) . The  necessary  conditions  for  minimization  are 
given  which  this  optimal  gain  must  satisfy;  it  is  shown  that  these 
conditions  result  in  a complex  two-point  boundary  value  problem. 

A second  optimization  problem  is  formulated  which  is  based  on 
the  restriction  to  non- learning  control  laws  which  are  precomputed; 
i.e.,  it  is  assumed  that  the  control  law  cannot  benefit  from  knowledge 
of  its  past.  Although  this  formulation  appears  to  be  much  weaker 
than  that  of  Problem  A,  it  is  shown  in  Theorem  2 that  if  steady-state 
solutions  to  the  two  problems  exist,  then  the  steady-state  solution 
to  Problem  A is  stabilizing  (in  the  sense  that  the  mean  square  value 
of  the  trajectory  is  exponentially  bounded)  if  and  only  if  the  steady- 
state  solution  to  Problem  B yields  a system  which  is  exponentially 
stable.  This  result  is  very  significant,  in  that  a Corollary  to  this 
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Theorem  solves  the  problem  of  finding  u robust  gain  foi  a set  of  linear 
systems  and  yields  an  explicit  procedure  for  its  calculation. 

The  last  Theorem  (Theorem  3)  of  the  Chapter  proves  that  the  steady- 
state  solutions  to  the  two  optimization  problem.s  are  identical.  This 
implies  that  not  only  does  the  procedure  mentioned  above  determine  a 
robust  gain  if  and  only  if  such  a gain  exists,  but  also  that,  the  steady- 
state  gain  is  optimal  with  respect  to  the  specified  quadratic  cost 
criterion. 


Problem  A. 

Consider  the 

system 

X . = A X + 

B , . , u 

(5.3.1) 

-t+1 1 

— k(t)  — t 

where 


X e r” 

(5.3.2) 

— t 

U ^ L R 

(5.3.3) 

— t 

k(t)  e I ={  0,1,2, ••• ,l} 

(5.3.4) 

1 is  an  indexing  set  for  the  possible  actuator  structures  { • 
where 


B.  e R 
~k 


n X m 


(5.3.5) 

)t(t)  is  a random  variable  with  sufficient  statistics  given  by  the 


Markov  transition  probabilities  where  the  matrix 

I - iPij> 


(5.3.6) 


is  a stochastic  matrix,  and  the  initial  probability  distribution  is 


,1 


^^0  = 


(5.3.7) 


Since  k(t)  is  assumed  to  be  a Markov  chain,  the  probability  vector 


Ti  ^ is  propagated  in  time  by 

T!  , = P TT*. 

- t+1 t 


(5.3.3) 


where  there  is  no  real-time  observation  with  which  to  update  . 

Consider  the  structure  space  fs , } , , indexed  by  I.  Define  the 

— k kr.I 

structural  trajectory  x ^ to  be  a sequence  of  element  k(t)  in  I which 


select  a specific  structure  B at  time  t, 

K \ u / 

x^=  (k(0),  kd),....  k(T-l)) 


(5.3.9) 


The  structural  trajectory  x ^ is  a random  Vcuriable  with  probability  of 
occurance  generated  from  the  Markov  equation  (5.3.8). 


p (x  _ ) = n 

T k(t),t 


where  the  control  interval  is 


(0,1,2, .. . ,T-1,t} 


(5.3.10) 


(5.3.11 ) 


for  the  finite  time  problem  with  terminal  time  T.  Then  for  a given 

T-1  — 

state  and  control  trajectory  (j^^  t^t-0  (5.3.1)  and  x^ 

T-1 

from  a sequence  of  controls  (u^  ^t-0  ' cost  index  is  to  be  the 

standard  quadratic  cost  criterion 


^ -t--t  ^ (5.3.12) 


The  admissible  controls  are  restricted  to  be  of  the  linear  feedback  form 


iit=  ^t^t 


* i,e,  ILq*  ® ...0)  or  (0  1 0...0)  or  . 


(5.3.13) 


. (0  0. . .0  1) 
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who  re  the  gain  matrix  G ^ ^ restricted  to  be  a function  only  of  time 
and  the  initial  conditions;  i.e.  , ^ cannot  depend  on  ^ . Ttie  objective 
is  to  minimize  over  tile  set  of  admissible  controls  tlie  exi>ectation  of 
(5.3.12),  where  the  expectation  is  taken  over  tiie  set  of  possible 
structural  trajectories 


*nt<  ^ n ^ 


(5.3.14) 


and  the  set  of  initial  conditions  x 


m w 

Thus,  the  optimal  control  law  u^  « should  minimize  the 


J = E[J  U 1 
T ‘ t'-O 


(5.3.15) 


over  the  set  of  admissible  controls. 

Since  the  structure  of  u ^ =*  G^x^  is  fixed,  the  problem  is  equiva- 
lent to  minimizing,  in  an  open- loop  sense,  the  cost  function 


E[\|]Tol 


I T T T T T 


T I 

X „ Q X _ IT  „ 
— T-*—  T ' — 0 


(5.3.16) 


with  respect  to  the  gain  matrix  , t«0, 1, . . . ,T-1.  Equation  (5.3.16) 
is  simply  obtained  by  substituting  equation  (5.3.13)  into  equation 
(5.3.14) . 


5.4  The  Method  of  Solution. 

The  matrix  minimum  principle  [Athans,411  will  be  used  to  determine 

* 

t)ie  necessary  conditions  for  the  existence  of  ij^  (or  equi valently , 

G*  ).  To  solve  the  problem  using  the  matrix  minimum  principle,  the 


r 


formulation  presented  in  the  last  section  must  be  converted  into  an 
equivalent  deterministic  problem.  For  this  purpose,  let  the  initial 
state  Xq  be  a zero  mean  random  variable  which  is  independent  of  any 
structure.  Let 


Lo  = E[XqxJ1tTq1  = EUolio 


be  the  convariance  matrix  of  x^  . 

Defining  the  covariance  of  x^  as 


then,  by  direct  calculation,  we  obtain 


(5.4.1) 


(5.4.2) 


= ...  P.  . P.  . •*•  P.  . TT. 

^ i ^0  i 4=0  i^  ^t-l^t-2  ^t-2^t-3  ^1^0 


t-1  t-2 


t-1  1 rt-1 

n (A+B  G )|E  n (A+B  G ) 
j=0  j M l:i=0  3 


Similarly,  if  we  define 


— i t ~ lk(t-l)=i,TT.Q  1 


(5.4.3) 


(5.4.4) 


then,  we  deduce  that 


li.t  - , to  - tb 


t-1  t-2  t-3 


n £( 

o'  1 3=0  j ■■  J 

I t-2  It 

n (A+B^  G ) (A+B.G^_j^)^ 

j=0  j ^ J 


(5.4.5) 
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Tho  matrix  T . ^ can  bo  defined  recursively  as 
-i.t 


^ . 

-3  ,t+l 


/ P..1I.  (A+B  . )?:  . ^ (A+B  . (5.4.6) 

TTj  ^ Di  D-t  -i.t D-t 


for  t > 1. 


£j,,  ■ 


(5.4.7) 


and  the  relation 


k ^ “ 


(5.4.8) 


is  obvious  from  direct  calculation. 


Remarkl:  At  this  stage,  an  equivalent  deterministic  problem  (Problem  AE) 


will  be  defined  with  state  (E  . . ).  . for  t>0  and  state  E at  t = 0. 

— i,t  1=0  —0 


The  system  dynamics  are  then  defined  by  equations  (5.4.6)  and  (5.4.7). 


Definition  (Problem  AE) : For  the  system  with  matrix  state  ^ ^ i=0 


for  t>0  and  for  t = 0 with  dynamical  equations  (5.4.6)  and  (5.4.7) 


and  matrix  control  G^  , minimize  the  equivalent  deterministic  cost 


over  (G^)J:J: 


= E- 
T X 


•io  I m 

+ '^oj 


tr(E^  (2  +G^RG^)]  + tr(E^2] 


(5.4.9) 
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Note  that  since  the  expectation  in  equation  (5.3.13)  is  over  all 
structural  trajectories  x and  the  initial  x^  also, 

jE  ,,  j (5.4.10) 

T T 

The  symbol  will  be  used  exclusively  in  the  future.  The  one-stage, 
or  instantaneous,  cost  at  time  t is 

tr(E^  (^  + G^RG^)1  (5.4.11) 

Problem  AE  is  completely  deterministic  in  the  state  ^^=0  ' —0 

and  control  . 

At  this  point,  the  minimization  will  be  decomposed  into  two  parts 
using  the  Principle  of  Optimality  (Athans  and  Falb,  21] . The  first 
minimization  is  over  the  interval  {l ,2, . . . ,T-l} , and  for  this  the  matrix 
minimum  principle  will  be  used.  The  resulting  solution  will  depend 
in  general  on  the  choice  of  G^  and  on  the  initial  conditions  and 

n-0- 

* 

Let  V (G  q)  be  the  optimal  cost  resulting  from  the  use  of  G^  and 
the  optimal  sequence  G , G^  # • • • * 5.^-1  interval  11, 2,..., T). 

The  second  minimization  is  then  over  G^  of  the  cost 

Jj.  = ^ (5.4.12) 

The  Principle  of  Optimality  states  that  these  two  minimizations 

* T-1 

result  in  the  minimizing  sequence  (G^)^_q 


for  Problem  AE. 
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From  (Athans,41I,  the  Hamiltonian  for  the  minimization  over 
{ 1,2, . . . ,T-l}  is 

"<  St' 


tr 


+ tr 


for  t e { 1 , 2, 3 , . . . ,T-l} 

where  the  costate  matrix  is  (S  . )V  „ . 

-j,t+l  j-0 


- j,t+l 


(5.4.13) 


Remark ; We  have  now  formulated  Problem  AE-1,  which  minimizes  the  accumu- 
lated cost  over  the  interval  {1,2,,..,t}  with  respect  to  the  sequence 
T-1 

(G^  matrix  minimum  principle  and  results  in  the  optimum 

* 

cost,  given  G ^ , V (G ^ ) . Problem  AE-2  is  then  the  minimization  of 
equation  (5.4.12)  over  G^  . 
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Define 


^0 


L ii,i  ^ 2.  + GqRGq 


(5.5.7) 


Then  from  equations  (5.5.6)  and  (5.5.7) 


= tr[I  „ S „ ] 
T —0—0 


T-1 


Thus,  the  cost  of  a given  sequence  ( \-0  length  T is 

•Jt  - «l£o^o'2o-=i 

I 

For  future  reference,  define  the  matrix  S . bv 

— i,t  ^ 


— i,t 


A — i,t 


ir. 

1 


(5.5.8) 


(5.5.9) 


(5.5.10) 


t-1 


and  note  that  equation  (5.5.2)  becomes 


T T T 

+ A S.  B.G^+G^BTS.  ^ ,Al 
- - D , t+1  - J-  t -t-]-],  t+1  - 


(5.5.11) 


From  the  Hamiltonian  minimization  necessary  condition 


3h 

3Gt 


= 0 


(5.5.12) 


the  following  relation  between  ^ , and  G is  obtained. 

j^t+1  “ t 

0 = RG  ^ TT.  E . ^ 


(5.5.13) 
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Remark ; At  this  point,  a two-point  boundary  value  problem  has  been 

defined  with  the  constraint  (5.5.13)  relating  equations  (5.5.2)  and 

(5.4.6).  Equation  (5.5.13)  is  not  explicitly  solvable  for 

because  Z . cannot  tse  factored  out  of  the  sum  over  j ; thus , it  cannot 
i # t 

be  used  as  a substitution  rule  in  the  other  two  equations.  At  this 

time,  the  solution  for  appears  intractable.  Thus,  although  necessary 

* 

conditions  for  the  existence  of  G^  , the  minimizing  gain,  have  been 

* 

established,  they  do  not  readily  allow  for  the  solution  of  G^  , and 
certainly  do  not  admit  a closed-form  expression. 
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5.6  rroblom  B:  The  Non-Switching  Solution. 

Although  the  ini'tho<.ioloijy  presented  in  Section  4 yields  the 
necessary  conditions  for  an  optimum,  these  conditions  are  not  analyti- 
cally illuminating.  In  this  section,  a second  optimization  problem 
is  formulated.  An  equivalent  formulation  was  presented  in  IBirdwell  & 
Athans,40).  The  solution  will  admit  a closed  form  expression  for  u^  . 
Although  this  solution  is  not  the  optimal  solution  for  the  first 
problem,  in  that  this  solution  does  not  necessarily  satisfy  the  neces- 
sary conditions  for  problem  AE , it  will  be  proved  that  the  two  solu- 
tions are  equivalent  in  the  sense  that  for  the  steady-state  solutions, 
as  defined  in  Section  7,  either  both  solutions  stabilize  the  system, 
or  neither  one  stabilizes  the  system.  Even  better,  it  will  be  proved 
that  the  steady-state  solutions  to  the  problems  are  identical. 

For  the  system  (5.3.1),  the  objective  is  to  minimize  at  each  time 
t the  weighted  sum,  with  respect  to  » of  expected  costs-to-go, 

given  the  control  — T * ~T  fori' t,  and  given 

that  the  structure  at  time  t-1  was  )c(t-l)  ■ i,  for  each  i. 

Formally,  let  C be  the  expected  cost-to-go,  given  , and 

)t(t-l)  at  time  t be  defined  as 

C(x^  , u^  , k(t-l),  t)  - + -t-  -t^ 


®)r(t)  I (5.6.1) 


where  * denotes  the  optimum  value,  and  u^  is  comi^uted  as 


u ■ arg  min  •C(t)^ 


Ht’^t  ‘^t’ 


> arg  min 

ii  “it  ‘-t’ 


n ‘_i  C(t) 


(5.6.2) 


(5.6.3) 


c (x^  ,k(t-l),t)  = c(x^  ,u^  ,k(t-l),t) 


where 


C(t)  = 


C(x^  •k(t-l)=0,t) 


C(Xj.  ,u^  ,k(t-l)=L,t) 


(5.6.4) 


(5.6.5) 


C(T)  = C*(T)  = 


Thus,  the  problem  is 


T 


T ■ 

X « Q X - 
— T — T. 


“t-tt  <St 


(X  ^ t-1 


+ E[C*(x^^j  ,k(t),t+l)|k(t-l)=ijl 

X*'  I T T 

min  7 TT.  |x^Qx^+u^Ru^ 

^ 1,.  1 I— t — t 1 


(5.6.6) 


(5.6.7) 


(5.6.8) 


From  the  formulation,  u^  is  non-learning  in  that  it  depends  only  on 


TT^  for  its  knowledge  of  the  past.  Let  C*  be  of  the  form 
C*  (x  ^ ,k(t-l)  ,t)  = t-t 

Then  for  t = T, 

— k,T  “ 2 


(5.6.9) 


(5.6.10) 


IP.IU.  m\m  I. 


\ 


116 


And  equation  (5.6.8)  becomes 


ST  T 

TT.  x^Qx^+u^Ru 

, — t--t  — t t 

t-l[ 


<ii,  * Sj  iit  >] 


(5.6.11) 


At  the  minimum,  differentiating  (5.6.11)  with  respect  to  u^  , we 
obtain 


0 = 


7 TI  . I R u ^ + 7 p..(B^S.  ....B.u^  +B^S.  ...Ax  l 

^ i^_l[— t 41^  -3-j,t+l-D-t  -D-3,ttl— tj 


(5.5.12) 


Solving  for  u^  , 


Ht=  - 


R + 


^ TT.  B.s.  ,.,,B.]  ^ TT.  B.S.  ...  Ax 


(5.6.13) 


and  hence  the  gain  matrix  is  given  by 


St  - -|£* 


(5.6.14) 


* * 

where  u ^ 


From  (5.6.11)  and  (5.6.4), 


T I T 

iLt^)c,t^t  = ^t 


*'T'  ♦ 

2 + 


^t 


(5.6.15) 


or,  since  (5.6.15)  holds  for  all  x^  , 


I *«T»  ♦ 

s.,.  * 2 + g^rg. 

— k,t  ^ — t 1 


Js  T ' T • * *>r  T ' 

7 p (A  s . A + A s . B . G . + G . B . S . 

^ ^jk -3,t+l-  - -3,t+l-]-t 


j,t+l- 


^ 1 


(5.6.16) 
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Thus,  (5.6.16)  proves  by  induction  that  equation  (5.6.9)  is  valid. 

Note  that  equations  (5.6.16)  and  (5.5.11)  are  identical. 

* 

Therefore,  the  unconditional  cost  of  , t=0, 1 , . . . ,T-1 , is,  from 


(5.5.9) 


= tr[I  Q S Q (Gq  ,G  ^ , . . . )1 


which  in  this  case  is  simply 


^T  S-oio  ‘-0  '-1  '•••'-T-1  ’-0 


(5.6.17) 


(5.6.18) 


The  matrices  G^  are  called  the  non-switching , or  non-learning  gains, 

* 

and  will  hereafter  be  denoted  G . The  label  G will  be  reserved  for 

— ns^  — t 

the  solution  to  equation  (5.5.13).  The  optimal  value  of  the  cost-to-go 
at  time  t=0  for  this  problem  will  be  called  the  non-switching  cost  index, 
and  is  given  by 


V*'  T T T 

= > Tl.x,S.,x,  + x„(Q  + G RG  )x- 

T m 

I / Tf.  <A+B  . G )^S  . , (A+B  . G ) 


(5.6.19) 


+ Q + G RG 

^ -nsg— nsQ 


(5.6.20) 


Note  that  if  G = G.  for  all  time  (i.e.,  if  the  solutions  to  the 

-nSfl  -t 

optimal  control  gain  problem  and  to  the  non-switching  control  problem 

are  the  same,  then  E [J  1 = • 

X ns„  T 

T 


Siiimnary : In  this  Section,  the  non-switching,  or  non-learning,  gains 

have  been  derived.  These  gains  are  called  non-switching  or  non-learning 
because  they  do  not  depend  on  the  past  trajectory  of  and  , but 
only  on  the  initial  probability  vector  over  I»  . It  was  further 
shown  that  if  the  solutions  to  Problems  AE  and  B were  identical,  then 


E [d  ] = 

X „ ns„  T 

— 0 T 


(5.6.21) 


I 


i 


I 

I 


I 
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5 . 7 Stability  and  the  Steady-State  Solutions. 

In  this  Section,  the  concept  of  stability  for  this  class  of 
systems  will  bo  precisely  defined.  From  this,  a natural  concept  of  a 
steady-state  solution  to  Problems  AE  and  B will  be  given,  and  a very 
strong  result  relating  the  solutions  to  the  two  problems  will  be 
proved . 

5.7.1  Stability  and  Cost-Stability. 

For  this  class  of  systems,  two  definitions  of  stability  will  be 
tendered.  The  first  is  the  usual  definition  of  mean-square  stability; 
the  second  definition,  that  of  cost-stability,  has  a strong  relation  to 
the  existence  of  solutions  to  the  infinite  time  versions  of  Problems  AE 
and  B. 


Definition  1;  (Stability) . G is  a constant  stabilizing  gain  if  and 
only  if  the  resulting  system  given  by  equation  (5.3.1)  and  repeated  here 


2it+l  = ^ ^)c(t)^t 


is  mean-square  stable: 
T 


0 


as  t > «>. 


(5.3.1) 


(5.7.1) 


Definition  2:  (Cost-Stability).  The  system  (5.3.1)  is  cost-stable 
if  and  only  if  the  scalar  random  variable 

CO 

(5.7.2) 

t=0  ^ ^ ^ 


I 


with  probability  one. 
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5.7.2  Pofinition  of  the  Infinite-Time  Cost. 

In  this  research,  the  infinite-time  problem  is  defined  as  a 
minimization  of 


lim  J 


(5.7.3) 


where  is  the  cost  function  for  the  corresponding  finite-time  problem. 


Th»!  sequences  which  solve  these  infinite-time  versions  of  Problems  AE 


and  B are  (G  ) and  (G  » respectively,  when  a solution  exists. 

■ t c * I'  t*ij 


A solution  will  exist  if  there  exists  a sequence  of  gains  for  which  the 
limit  in  equation  (5.7.3)  exists.  This  definition  of  the  infinite-time 
problem  is  chosen  rather  tlian  the  definition  requiring  a minimization 
of  the  average  cost  per  unit  time 


T T 


(5.7.4) 


because  there  is  a direct  correlation  between  the  boundedness  of 


over  all  T for  a constant  sequence  of  gains  G and  mean  square  stability 
of  the  system  (5.3.1).  It  is  necessary,  however,  to  prove  that  the 


set  of  pr<'blems  for  which  is  bounded  for  some  sequence  of  gains  is 


not  vacuous.  This  fact  is  demonstrated  by  any  of  the  convergent  non- 
switching gain  examples  in  Section  10. 

As  further  demonstration  of  the  validity  of  using  equation  (5.7.3), 
note  that  if  0 < < <*>,  then  the  cost  per  unit  time  has  a non-zero 

steady-state  value,  which  implies  that  the  system  (5.3.1)  is  not  mean- 
square  stable  since 


J,  =•  trlE  (O  + G RG  ) 1 
1 — ss  * — ss ss 


(5.7.5) 


where  E and  G are  the  steady-state  values  of  E ^ and  G ^ , when 
— ss  — ss  — t — t 


they  exist,  and,  since  £ + G RG  is  positive  definite,  Z 0. 

S S s s ■ s s 


121 


pWHPIPIVVPOTPI^pilPirni 


5.7.3  Bounded  Cost  and  Mean-Square  Stability. 

In  choosing  equation  (5.7.3)  as  the  basis  for  the  definition  of  an 
infinite-time  problem,  a major  requirement  was  that  the  existence  of 
an  infinite-time  solution,  namely  of  a sequence  of  gains  which  yields  a 
finite  cost  in  equation  (5.7.3),  imply  mean-square  stability.  For 
the  case  where  the  sequence  is  constant,  the  following  result  is 
proved . 

OO 

Theorem  1 ; A constant  sequence  of  gains  mean-square  stabiliz- 

ing if  and  only  if  there  exists  a bound  B < “>  such  that 

< B for  all  T (5.7.6) 

T 

Proof : See  Appendix  5.1. 

OO  . . . 

Remark:  For  a sequence  J^<B<“>\/T  implies 

OO 

mean-square  stabilizing,  but  (G  ^)  mean-square  stabilizing  does  not 
imply  is  bounded  for  all  T. 

Proof : See  Appendix  5.2. 

5.7.4  Cost-Stability. 

As  yet,  the  definition  of  cost-stability  has  not  been  utilized. 

In  this  Subsection,  it  will  be  shown  that  the  system  described  by 

OO 

equation  (5.3,1)  is  cost-stabilized  by  a sequence  of  gains  (G^)^_yif  and 
only  if  J is  finite-valued  for  this  sequence.  One  direction  of  this 
result  is  proved  in  the  following  theorem. 

00 

Theorem  2:  Any  sequence  which  j < °°  cost-stabilizes  (5.3,1) 

with  probability  1, 

Proof ; See  Appendix  5.3. 


VPVMHIIlim 
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The  other  direction  of  this  result  is  obvious:  If  a sequence 

OO 

(fi^)^_Qis  cost-stabilizing  with  probability  one,  then  the  random  cost, 
given  by  equation  (5.7.2),  is  finite  except  on  a set  of  structural 
trajectories  of  measure  zero.  (The  appropriate  measure  on  this  set  is 
given  in  the  proof  to  Theorem  2.)  Since  the  expected  cost  J is  the 
integral  of  equation  (5.7.2)  with  respect  to  the  probability  measure 
on  the  set  of  structural  trajectories  (see  Appendix  5.3),  then  J is 
finite. 

Thus,  the  cost-stability  and  the  existence  of  an  infinite-time 
solution  are  equivalent. 

5.7.5  Equivalence  of  Problems  AE  and  B. 

The  first  major  result  of  this  Chapter  u*ll  now  be  stated.  This 
result  establishes  a strong  equivalence  between  the  solutions  to 
Problems  AE  and  B. 

00 

Theorem  3:  A cost-stabilizing  solution  (G  ) „ exists  if  and  only  if 

—ns  t=0  ^ 

* OO 

there  exists  a cost-stabilizing  solution  ' assuming  Tr^>  0 for 

all  i and  Zq>0. 

Proof  : See  Appendix  5.4. 

Remar)c  1 : This  result  provides  a computationally  feasible  methodo- 

00 

logy  for  arriving  at  a sequence  of  gains  (G  ) , which  cost-stabilize 

— ns^  t=0 

the  original  system  (5.3.1)  with  probability  1,  whenever  such  a se- 
quence exists.  The  coupled  matrix  equations  of  Problem  B (5.6.16)  can 
be  iterated  backward  in  time.  If  the  weighted  sum  with  respect  to  the 
erqodic  distribution  converges,  then  the  resulting  sequence  of  gains 
cost-stabilizes  the  system  (5.3.1)  with  probability  one. 


A steady-state*  solution  to  optimization  Problems  AE  and  B can 
exist  only  if  there  exists  a steady-state  probability  distribution  n 
over  set  of  possible  configurations  indexed  by  I such  that 


Ti_  =*  P Tf  (5.7.7) 

and 

lim  JT  . = >1  (5.7.8) 

t ♦<» 

From  equation  (5.7.7),  it  is  apparent  that  for  to  exist,  the  matrix 
£ must  have  an  eigenvalue  at  1,  and  Ti_  must  be  in  the  subspace  spanned 
by  the  eigenvectors  of  P corresponding  to  that  eigenvalue.  The  fol- 
lowing lemma  states  precisely  when  n exists. 

Lemma  1 : Ti^  exists  if  and  only  if  one  of  the  following  three  conditions 
is  satisfied  for  each  diagonal  element  of  the  Jordan  normal  form  _A 
of  P,  where 

P = T A T~^  (5.7.9) 

a,  p,  0 

aa 

0 

For  each  i , 


r 


I 

n 

j. 

■ 
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i)  01,  < 1 
' 1 ' 

ii)  a.  = 1 
1 


iii)  |a^|=  1,  / 1,  (t"^  = 0 


Proof:  Obvious. 


5. 7. 6.1  Steady-State  Solution  to  Problem  AE. 

* * 

Note  that  for  Problem  AE,  initially,  the  gains  , G ^ 

* * 

will  depend  on  and  near  the  final  time,  the  gains  ...  G^  2 ' 

will  depend  on  a time-varying  S^  Thus,  the  steady-state  solution  for 
Problem  AE  is  defined  as  the  limiting  solution  to  equations  (5.4.6) 
(5.5.2)  and  (5.5.13)  at  time  t,  first  as  T^  and  then  as  t-»<®,  if  this 
limit  exist.  The  steady-state  values  for  B»  S * and  , when 
they  exist,  satisfy  the  following  equations: 


E.  = - 


> p..  tr.  (A+B,  G ) E.  (A+B.G) 
TTj  ^ *^31  1 j - -I 3- 


(5.7.11) 


[T  IT  T T T 

Q + G RG+  > p..-  [A  S.A  + G B.S.B.G  + A S.B.G 
^ ^4^  ^317T^  -3-  - -3-j-3-  - -3-j- 

T T I 
+ G B ,S  .A) 

--3-3-J 

0 = RG  7 IT.  E.  + > — (B^S,B,G  + bTs.AI  7 p . . IT . E. 
--  H " i4r5  1 -ij 


(5.7.12) 


|(B^S,B,G  + bJs. 

-3-j— )-  -j— 

A — V 

(5.7.13) 

which  are  the  limit  of  equations  (5 . 4 . 6) , (5. 5. 2) , and  (5.5.13),  given 

that  the  limiting  solution  E,  and  G exist,  where  satisfies 

—3  » t — t 

equations  (5.7.7)  and  (5.7.8).  The  cost  of  this  steady-state  solution 


is 

J ” lim 

T 

T-kd 

as  in  equation  (5.7.3). 


(5.7.14) 
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5. 7. 6. 2 Steady-State  Solution  to  Problem  B. 

The  solution  to  Problem  B depends  on  its  past  only  through  the 
probability  distribution  Ti(t)  over  the  structure  index  set  I. 

Therefore,  to  develop  the  steady-state  solution,  let  the  initial  pro- 
bability distribution  equal  the  steady-state  value  7t_  from  equations 
(5.7.7)  and  (5.7.8).  Then  the  steady-state  solution  can  be  defined  as 

the  limit,  when  it  exist,  of  the  gain  G calculated  for  the  problem 

— ns 

ending  at  time  T,  and  of  the  solutions  to  the  coupled  Riccati-li)ce 

equations  (5.6.16),  S.  , as  the  final  time  approaches  infinite.  Let 

i , 0 

G (T)  and  S . „(T)  be  the  solutions  at  time  zero  for  Problem  B with 

-nsg  -1,0 

final  time  T.  Then 

G = lim  G (T)  (5.7.15) 

"ns  — ns^ 

<p-w  0 

• • 

S.=  lim  S . (T)  , iei  (5.7.16) 


When  the  limits  exist.  The  steady-state  solution  is  said  to  exist 
whenever  the  limits  of  equation  (5.7.16)  exist.  If  these  limits  exist. 


then  G and  S . must  satisfy,  from  equations  (5.6.14)  and  (5.6.16). 
— ns  — 1 


G 

— ns 


- R + ^ TT.  b"^  S.  B.  ^ TT.b'!'  S.  a 


(5.7.17) 


= Q + G R G 
*■  —ns ns 


T ' T ' 


T T ' 

+ G B.S..A 
ns  — ns-j-g  — 


T T ' 

+ G B,S.B,G  ) 
— ns  -g-g-j  — ns 


(5.7.18) 


The  cost  of  this  steady-state  solution,  given  Jt,  is,  when  the  limit 
exists 


F 

f 


1 


J 

ns 


lim  J 

T-*oo 


ns„ 


T 


X 


TT  . S . X 
1—1  — 
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(5.7.19) 


5.7.7  The  Possibility  of  Limit  Cycles.  ; 

f 

The  discussions  in  the  last  Section  do  not  rule  out  the  possibi-  j 

t 
I 

lity  of  limit  cycles  in  an  infinite-time  solution.  In  Problem  B, 
the  expected  cost  is  directly  computable  from  a set  of  coupled  Riccati- 
like  equations  (5.6.16),  as  is  the  non-switching  gain  (5.6.14).  If 
tliese  coupled  matrix  equations  converge  whenever  the  solution  is 
bounded,  then  the  non-switching  gain  is  always  directly  computable  when 
it  exists.  Boundedness  implies  convergence  of  the  expected  cost 
(Lemma  2) ; however,  the  possibility  of  the  existence  of  a limit  cycle 
in  the  solution  to  equation  (5.6.16)  is  not  ruled  out.  It  is  con- 
jectured, but  not  proved,  that  such  a limit  cycle  cannot  exist. 

* 

Lemma  2;  If  the  expected  cost  for  Problem  A is  bounded,  then  it 
converges. 

Proof ; See  Appendix  5.5. 

* 

Since  E (J  1 = >J„,  J also  converges. 

X ns„'  T ns„  ^ 

T T 


1 
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* 

S.8  Equality  of  and  G 

In  this  Section  it  will  be  shown  that  when  a steady-state  G and 


— ns 

* * * 

G exist,  with  finite  cost  J and  J , the  gains  are  equal.  This 

result  is  extremely  important  in  that  it  yields  a method  of  calculating 
the  steady-state  solution  to  a two-point  boundary  value  problem  as  the 
limiting  solution  to  an  equivalent  (in  the  steady-state)  single  boundary 
value  problan.  it  is  taken  as  a working  hypothesis  in  this  S3ction  that 
both  problems  have  a steady-state  solution  and  that  the  ergodic  distribu- 
tions of  Tt_  and  E ^ , for  all  i,  exist.  Then  the  steady-state  cost  of  the 
optimal  problem  is 


J33  = trlE^Ca^G^RG^  )J  + > trlE.S.) 


(5.8.1) 


1=1 


For  any  constant  gain  G for  which  the  limits  exist,  the  value  would 
be 


(G)  = trlE^  <2  + G RG  )]  + 


tr(E_.  (G)S  . (G)l  (5.8.2) 


= triE  Q (g  + G^  RG  )1  + trl  (A+B  ^ G)E  Q (A+B  . G)^  ^ (G)  ] 


= tr 


I T 

( Q + G R G + 


SS  * ^ 

Similarly,  equation  (5.8.1)  becomes 


(AtB  . G)  S . (G) (A+B  . G)} 
1-  -1  x-{ 


(5.8.3) 


(5.8.4) 


J = tr 
ss 


Soja  * Css'o  ♦ fi+sUo'} 


(5.8.5) 


For  tfie  non-switching,  or  non-learning  problem,  the  steady-state  cost 

I 

for  any  G for  which  the  S converge  is,  given  x^  , 
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J (G)  = x^(Q  + G^RG)x_  +E 

ns  — —0*“  —O —0 


ss 


(5.8.6) 


= iij  <2  + g'^RG)Xq 


TT.  (A+B  . G)  S . (G)  (A+B  . G)X  „ 
1 1—  —1  — 1 0 


(5.8.7) 


Taking  expectations  with  respect  to  x^  , 


ns  — 
ss 


= tr[E  Q (2  + G RG  )] 


+ > {G)(A+B^G)] 


(5.8.8) 


or. 


EiJns  (G) 
ss 


= J (G) 
ss  — 


(5.8.9) 


Thus,  the  costs  are  equivalent  for  any  G for  which  the  equations 
converge . 


By  Lemma  3,  if  the  non-switching  expected  cost  is  bounded  for  a single 
G,  then  the  equations  converge;  i.e.,  there  can  be  no  limit  cycle. 

Lemma  3 : For  a given  gain  G,  if  the  expected  cost  J,p(G)  is  bounded 
then  it  converges. 

Proof ; See  Appendix  5.6. 

Thus,  either  equation  (5.8.9)  holds,  or  both  costs  are  infinite.  There- 
fore, if  the  cost  is  finite  for  any  single  G , then  there  exists  a G 

— —opt 

which  minimizes  both  costs.  Furthermore,  given  that  G^^  (T)  converges, 

G (T)  G ^ as  T -♦■  <».  This  result  with  an  extension  is  stated  in 
— ns^  —opt 

Theorem  4. 


• )4*'l 
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Theorem  4:  Assume  the  values  G.  (T) , G (T) , S . (T) , S , (T) , and 

— t ~i»t 

T.  . converge.  Then 

A)  G (T)  -►  G as  T which  minimizes  equation  (5.8.9/. 

— ns^  —opt 

* 

B)  G = G , where  G is  the  steady-state  value  of  G (T) , 

— ns  - —ns  -ns^ 

* * 

and  G is  the  steady-state  value  of  G^  (T)  : 


* * 

lim  lim  G (T)  = G 

t Kx)  y-*<x) 


(5.8.10) 


Proof:  See  Appendix  5.7. 

Discussion:  The  result  of  Theorem  4 B)  gives  a direct  computational 

* 

procedure  for  calculating  the  optimal  steady-state  gain  G as  the 
limiting  gain  G^^  . There  are,  however,  still  some  open  questions 
concerning  the  existence  of  limit  cycles  in  the  calculation  of  • 

oo 

Theorem  3,  however,  guarantees  cost-stability  using  ^t=0  * 

cost-stabilizing  sequence  of  gains  exists. 
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5 . 4 Robustness. 

The  original  problem  (Problem  A)  can  be  formulated  in  such  a way 

OO 

that  the  sequence  (G  ) ^ will  cost-stabi lize  a set  of  linear  systems 

— ns^  t=0 

with  different  actuator  structures  individually  whenever  such  a stabiliz- 
ing or  robust  gain  exists. 

Definition  3;  A gain  G is  robust  if 

X = (A  + B.  G)x  ^ (5.9.1) 

— t+1  — — 1 

is  stable  for  all  )t.  This  is  the  same  as  requiring  the  matrix  (A+B^j^G) 
to  have  eigenvalues  inside  the  unit  circle  for  all  k. 


Corollary  1:  For  the  set  of  L+1  systems 


^t>l 


(5.9.2) 


with 


P = I 


(5.9.3) 


n , 
3 


1 

L+1 


(5.9.4) 


OO 

if  a robust  gain  exists,  then  (G  ) ^ ,. 

— ns^  t=0 


is  a stabilizing  sequence  for 


(5.9.1)  for  each  k,  and  if  the  gains  G^^  (T)  converge,  then  G^^  is  a 


robust  gain. 

Proof ; For  the  expected  cost  to  be  finite,  for  any  G , G must  be 
robust,  since  each  structure  is  equally  likely  and  no  structural  changes 

* OO 

can  occur.  Therefore,  if  a robust  G exists,  then  certainly  (G^  ^ t=0 

00 

will  be  stabilizing,  and  by  Theorem  3,  so  will  (G^^  ^t=0  ' 


(T)  converges  as  T the  G 

t 


G 

— ns 


will  be  robust  since  it  will  have 
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finite  cost  J(G  ),  which  implies  stability,  in  this  case,  for  all 
— ns 

k e I. 

Q • E . D • 

Discussion : With  Corollary  1,  a specific  existence  problem  for  robust 
linear  gains  is  solved.  Existence  of  a robust  gain  is  made  equivalent 
to  the  existence  of  a finite  cost  infinite-time  solution  to  Problem  B, 
which  is  readily  computable  from  equations  (5.6.14)  and  (5.6.16). 


5.10  Examples. 
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In  this  section,  two  examples  are  presented  to  illustrate  the 
non-switching  gain  computational  methodology.  Example  5.1  is  ana- 
logous to  Example  3.1  of  Chapter  3;  it  demonstrates  the  effect  of 
component  reliability  on  system  stabilizability  with  a non-switch- 
ing gain  control  law.  The  first  case  of  Example  5.1  is  not  conver- 
gent; the  second  case  is  convergent.  The  only  difference  between 
the  two  cases  is  the  reliability  of  the  actuators.  Case  i)  corresponds 
to  Case  ii)  of  Example  3.1;  Case  ii)  corresponds  to  Case  iii)  of 
Example  3.1.  Neither  case  results  in  a robust  control  law,  but  ro- 
bustness is  not  possible  because  the  system  is  uncontrollable  in 
structural  state  3.  As  an  aside,  it  is  interesting  that  the  "optimal" 
non-switching  gain  in  Case  i)  ignores  state  x^;  the  system  is  decoupled 
in  that  there  is  no  interaction  between  x^^  and  x^.  Since  state  x^ 
has  stable  dynamics,  and  the  dynamics  of  state  x^^  are  unstable,  the 
entire  control  effect  is  concentrated  on  state  x^. 

The  computer  routines  which  are  used  in  the  calculation  of  the 
non-switching  gain  solution  are  listed  in  the  Appendix.  The  primary 
subroutine  is  AIM;  it  calls  WEIGHT.  Any  other  routines  which  are 
used  are  from  the  standard  ESL  subroutine  library. 
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The  system  is 


^it+1  = ^ ^k(t) 


e {0,1,2, 3} 

The  cost  to  be  minimized  is 


f^,t  ’‘2,t' 


r 


II 


j ; 


Example  5.1,  Case  i) 


Pf  = -1. 
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Nan-Convergent;  but  gain  converges  at 
-1.246  0.0 

-1.039  0.0 


G 

— ns 


Stability: 


J 

Configuration 

Stable 

0 (B  _ ) 

no 

— u 

1 (B  1 ) 

yes 

2 (§2  ) 

yes 

3 (B  . ) 

no 

— J 

coupled  Riccati 

equations 

that  since  state  has  stable  dynamics,  the  convergent  non-switching 


gain  concentrates  on  stabilizing  Xj^,  which  is  open-loop  unstable. 


From  the  above  stability  table,  the  control  law 


u . = G X ^ 

— t — ns— t 


stabilizes  only  configuration  states  1 and  2;  since  the  configuration 
has  a high  probability  of  being  in  state  0 (unstable),  the  cost  diverges. 


Example  5.1,  Case  ii) 

P,,  “ .01,  p = .98 
f r 


m _ 

.9799 

% 

.009999 

. 009999 

”2 

.0001020 

^ - 

3 

Convergent  Coupled  Riccati  Equations. 

-.7563  .1266 

-.8070  -.1784 

Stability : 
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-ns 


Configuration 

Stable 
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<5^0  > 

yes 

1 

(B,) 

no 

2 

(B2) 

no 

3 

<13) 

no 
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Interpretation:  With  mc>re  reliable  actuators,  the  non-switching  gain 
expends  less  force  on  th*e  stabilization  of  configuration  states  1 and  2 
(unstable);  since  configuration  state  0 is  stabilized,  and  the  system 
has  a (relatively)  higher  probability  of  being  in  configuration  state  0 
than  in  Case  i) , the  non-switching  coupled  Riccati  equations  converge, 
resulting  in  a finite  cost. 
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Example  5.2  uses  the  same  system  dynamics  as  In  Example  5.1; 
however,  only  structures  0,1  and  2 (the  controllable  structures)  are 
considered.  The  configuration  dynamics  are  modeled  as  being  in  any 
structural  state  with  equal  probability  of  occurance  initially  and 
remaining  in  that  state  forever;  this  model  is  illustrated  graphically 
in  Figure  5.1. 

The  state  dynamics  are 


^t+l  = ^^t  ^ Sk(t)^t 


k(t)  c {0,1,2} 

The  cost  to  be  minimized  is 

J = E 


-t  ° '’‘l,t  ’‘2,t^ 


The  non-switching  methodology  yields  a robust  control  law  of  the 


form 


u ” G X ^ 

— t — ns  — t 
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Example  5.2; 
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Stability: 


112.8 
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-.01444 
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6.835 


Configuration  Stable 


0 <«0> 

1 (B  ^ ) 

2 (B^  ) 


yes 
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Robust:  yes 


The  non-switching  solution  converges  for  the  system  in  Example  5.2, 

and  the  three  resulting  configurations  are  stabilized.  Therefore  G 

— ns 

is  a ^bust  gain.  Had  the  solution  not  converged,  by  Corollary  1 of 
Section  9,  no  robust  gain  would  exist. 

The  apriori  expected  cost  (before  the  configuration  state  is 
known)  is,  given  x 

T 

J = X CX 


5.11  Summary . 


In  this  Chapter,  an  optimization  problem  was  defined  on  linear 
systems  with  variable  actuator  configurations  and  quadratic  cost  criteria 
The  objective  of  this  approach  was  to  compute  apriori  a sequence  of 
gains  to  be  used  in  linear  feedback  control  which  do  not  depend  on 
any  on-line  information  about  the  process.  These  gains  were  to 
both  stabilize  the  overall  system,  accounting  for  the  various  possible 

structures  and  minimize  the  expected  value  of  the  quadratic  cost  crite- 

* 

rion,  where  the  expectation  is  taken  over  the  possible  sequences  of 
actuator  configurations.  This  solution  depends  on  both  the  perfor- 
mance, and  on  the  reliability  of  the  various  structures,  as  represented 
by  the  Markov  transition  probabilities  between  structures. 

The  matrix  minimum  principle  [Athans,41]  was  used  to  establish  the 
necessary  conditions  for  optimality  of  a solution  to  an  equivalent 
deterministic  problem  to  that  described  above,  known  as  Problem  AE  in 
the  Chapter.  These  conditions  unfortunately  do  not  yield  an  analytic 
solution  for  the  gain  sequence,  but  instead  yielded  an  ill-posed  two- 
point  boundary  value  problem  which  must  be  solved  numerically  (Section  5) 
Therefore,  a second  problem  (Problem  B)  was  formulated  which  was  solvable 
analytically  using  dynamic  programming  (Section  6) . This  solution  has 
identical  cost-stabilizing  properties  to  the  solution  of  Problem  AE, 
but  has  the  advantage  of  being  directly  computable. 

The  steady-state  solutions  to  the  inf inite-time  versions  of  both 
problems  were  defined,  when  they  exist,  and  it  was  proved  that,  in  addi- 
tion to  the  equivalent  stabilizing  property  of  the  two  solutions,  the 
steady-state  values  are  identical,  and  this  value  is  the  same  as  the 
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i 

1 


optimal  constant  gain  which  minimizes  the  expected  cost  over  the  infinite 
time  interval. 

In  addition,  the  general  robustness  question  of  when  one  gain  can 
stabilize  a set  of  linear  systems  with  different  actuator  configurations 
was  formulated  in  the  context  of  Problem  A and  was  solved  by  Problem  B. 
Thus,  a test  for  when  a robust  gain  exists  can  be  performed  by  iterating 
a set  of  coupled  matrix  Riccati-like  equations  and  testing  for  converg- 
ence of  a function  of  the  solutions.  If,  in  addition,  the  individual 
solutions  converge,  then  the  robust  gain  which  minimizes  the  expected 
quadratic  cost  index  can  be  calculated  directly.  It  was  noted  that  the 
extension  to  systems  with  variable  dynamics  (variations  in  A) , as  well 
as  variable  actuator  structure,  is  trivial  as  long  as  the  dimension  of 
the  state  is  constant. 

The  major  applications  of  this  work  are  in  the  calculation  of  a 
robust  gain  for  a set  of  linear  systems  and  in  the  calculations  of 
stabilizing  gains  for  systems  with  variable  structure,  such  as  occurs  in 
failure,  repair,  or  reconfiguration.  A second  application  will  be 
covered  in  the  next  Chapter  and  involves  using  these  calculations  in  a 
computer-aided  design  procedure  for  the  determination  of  the  relative 
effectiveness  of  various  redundant  component  configurations. 


iLi 
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CHAPTER  6 

COMPUTER-AIDED  DESIGN 

6. 1 Introduction. 

In  this  Chapter,  two  specific  applications  of  the  non-switching 
gain  methodology  to  computer-aided  design  are  presented.  Example  6.1 
illustrates  the  usefulness  of  the  non-switching  gain  methodology  in 
the  selection  of  an  actuator  design.  Five  possible  designs  are 
analyzed  using  the  non-switching  gain  calculations  as  a basis  for  ranking 
the  designs  with  respect  to  their  expected  performance.  Example  6.2 
compares  two  actuators,  of  which  one  is  more  reliable,  but  less 
effective  (in  that  it  incurs  a greater  cost  for  the  same  action)  than 
the  other.  Three  cases  with  various  actuator  reliabilities  are  presented 
as  a study  of  the  trade-off  between  actuator  reliability  and  effective- 
ness. 

These  two  examples  are  intended  to  demonstrate  the  usefulness  of 
the  non-switching  gain  methodology  in  design  studies.  No  general  method- 
ology for  computer-aided  design  using  the  results  presented  in  this 
report  is  presented.  Instead,  tools  are  presented  which  can  be  used  in 
the  computer-aided  design  of  system  configurations. 

6.2  The  Design  Decision. 

A designer  often  has  many  means  of  achieving  a desired  goal; 
however,  no  unified  methodology  exists  which  can  be  used  to  choose  a 
given  design  that  is  ’’better"  than  any  other.  At  best,  a set  of  tools 


i 


can  be  developed  which  are  applicable  to  specific  situations  and  classes 
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of  systems.  Of  these  tools,  all  that  are  presently  available  evaluate 


a system  either  on  the  basis  of  performance  or  on  the  basis  of  reliabil- 
ity. The  methodologies  described  in  this  report  optimize  a performance 
index  which  depends  on  both  system  reliability  and  system  performance. 
Therefore,  it  is  logical  to  apply  these  methodologies  to  the  computer- 
aided  design  of  system  configurations. 

Example  6.1  is  an  aid  in  the  design  of  a linear  system  for  which  the 
state  dyncunics  are  fixed,  but  the  actuator  configuration  is  to  be  at 
most  two  actuators  (one  level  of  either  component  or  functional  redundancy) 
chosen  from  two  types  of  actuators.  The  system  in  Example  6.1  is  de- 
fined by 


2Lt+i=  ^)c(t)iit 


(6.2.1) 


)i(t)  e I (6.2.2) 

T 

where  x = [x  , x , x ] . In  Cases  i)  and  ii) , I = {o,l}  ; 

u XfC 

in  Cases  iii) , iv) , v) , I = {o,l,2,3}.  The  cost  to  be  minimized  is 

(6.2.3) 


Jo,  = E 

T 


CO 

ST  T 

Xt^-t'*’  -t--t  I - 


The  cost  of  each  actuator  (labeled  b^  and  ) is  to  be  the  quadratic 
cost  incurred  by  the  control  input  to  that  actuator.  These  costs  are 
represented  by  the  quadratic  weights  r^  and  r^^,  respectively,  and  are 
equal  in  Example  6.1.  The  actuators  act  on  different  states  of  the 
system;  actuator  b^  applies  the  control  force  to  state  x^#  while  b^^ 
applies  the  control  force  to  state  x^.  Each  actuator  can  fail  to  an 
actuator  with  zero  gain,  0^.  Repair  constitutes  replacement  of  the 
failed  component  with  a new  actuator,  identical  to  the  original  ac- 
tuator. The  repair  action  is  modeled  using  a Mar)cov  transition  pro- 
bability p^,  the  probability  of  repair  per  unit  of  time.  The  actuators 


i! 


! ;• 

^ i 
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have  identical  probabilities  of  failure  and  repair  per  unit  time,  p^  and 
p^,  respectively.  The  five  possible  actuator  configurations  are,  in  the 
order  in  which  they  are  presented  in  Example  6.1, 


= 

l^ol 

(6.2.4) 

II 

ml 

■Ql 

(6.2.5) 

' B^  = 

i 

1^0  1 

^ol 

(6.2.6) 

tl 

.Ql 

^ll 

(6.2.7) 

b'  = 

ko\ 

^ll 

(6.2.8) 

1 2 

Configurations  B and  B have  two-state  configuration  dynamics  directly 

defined  by  the  failure  and  repair  probabilities  per  unit  time.  Con- 
3 4 5 

figurations  B , B and  B have  four-state  configuration  dynamics  re- 
presented graphically  by  Figure  3.2  of  Chapter  3,  Section  5.  It  is 
not  immediately  obvious  from  the  configurations  and  the  state  dynamics 
which  configuration  is  optimal.  When  a non-switching  gain  control  is 
used,  the  expected  steady-state  cost,  given  by  equation  (5.7.3),  is 
a measure  of  the  expected  performance  of  each  configuration,  and  can  be 
used  to  rank  the  five  configurations  in  order  of  system  effectiveness. 
System  effectiveness  is  a measure  of  the  expected  performance  of  a 
system,  taking  into  account  all  postulated  modes  of  operation.  There- 


fore, in  Example  6.1,  the  non-switching  gain  and  expected  cost  is  com- 
puted for  each  of  the  five  design  configuraitons . 
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f 
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stability; 


Configuration 

Stable 

“'“o’ 

yes 

1 (B  J> 

no 

Interpretation:  The  steady-state  non-switching  gain  exists;  it 

stabilizes  configuration  0 does  not  stabilize  configuration 

1 (B^)  . Since  the  probability  of  being  in  configuration  0 (stable) 

( tTq  ) is  much  greater  than  the  probability  of  being  in  configuration  1 

(unstable)  (Tr^^),  the  system  configuration  is  stabilized  using  tlie 

non-switching  gain  G in  the  control  law 
— ns 

u ^ = G X ^ 

— t — ns— t 
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Example  6.1  Case  ii) 
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Convergent  Coupled  Riccati  Equations: 
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Expected  cost 


T 

X C X 


Stability: 


Configuration  Stable 


Interpretation:  The  steady-state  non-switching  gain  exists;  it 
stabilizes  configuration  0 (B^),  but  does  not  stabilize  configuration 
1 ).  Since  the  probability  of  being  in  configuration  0 (stable) 

( TT^)  is  much  greater  than  the  probability  of  being  in  configuration  1 
(unstable)  (Hj^)  , the  system  configuration  is  stabilized  using  the 
non-switching  gain  G in  the  control  law 

— t “-ns— t 
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Convergent  Coupled  Riccati  Equations: 


-2.469  -.1279  -.8983 

-2.469  -.1279  -.8983 


153.1  32.81  48.01 

32.81  9.050  10.92 

48.01  10.92  19.03 

154.4  32.88  48.48 

32.88  9.054  10.95 

48.48  10.95  19.2a 

• « 

154.4  32.88  48.48 

32.88  9.054  10.95 

48.48  10.95  19.20 

155.8  32.95  48.96 

32.95  9.058  10.97 

48.96  10.97  19.39 


153.2 

32.82 

48.02 

32.82 

9.050 

10.92 

48.02 

10.92 

19.04 

Expe..'ted  cost  “ x C x 
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Stability: 


Configuration 


0 


1 (B  1 ) 


2 (B2  ) 

3 (B  3 ) 


Stable 

yes 

no 

no 

no 


Interpretation:  The  steady-state  non-switching  gain  exists;  it 
stabilizes  configuration  0 ) , but  does  not  stabilize  configurations 

1,2, or  3.  Since  the  probability  of  being  in  configuration  0 (stable) 
(TT^)  is  much  greater  than  the  probability  of  being  in  any  other  con- 
figuration (TT^,  i=l,2  or  3)  (unstable),  the  system  configuration  is 

stabilized  using  the  non-swithcing  gain  G in  the  control  law 

“ns 


u ^ = G X ^ 
— t — ns— t 
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Convergent  Coupled  Riccati  Equations: 
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Expected  cost  = x C x 
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stability: 


Configuration  Stable 


0 

1 (B^  ) 

2 (B^  ) 

3 (B  3 ) 


Interpretation:  The  steady-state  non-switching  gain  exists;  it 

stabilizes  configuration  0 (B^q)#  hut  does  not  stabilize  configurations 
1,  2,  or  3.  Since  the  probability  of  being  in  configuration  0 (stable) 
(TT^)  is  much  greater  than  the  probadsility  of  being  in  any  other  con- 
figuration (tt^,  i=l,2  or  3)  (unstable),  the  system  configuration  is 
stabilized  using  the  non-switching  gain  in  the  control  law 

u ^ = G X ^ 

— t — ns  — t 


Convergent  Coupled  Riccati  Equations: 


G 

— ns 


Expected  cost 


r-3.815 

-.1312 

1-2.956 

-.5815 

126.5 

24.86 

24.86 

7.066 

32.32 

6.842 

ri28.4 

24.93 

1 24.93 

7.096 

00 

00 

CN 

6.863 

127.3 

25.01 

25.01 

7.097 

32.72 

6.921 

129.2 

25.08 

25.08 

7.100 

33.28 

6.942 

126.5 

24.86 

32.33 


-1.106 

-1.486, 

32.32 

6.842 

10.69 

32.88 
6.863 
10.85 

32.72 

6.921 

10.89 

33.28 

6.942 

11.05 


24.86 

32.33 

7.067 

6.843 

6.843 

10.69 

I 


stability : 


Configuration  Stable 

0 (B Q ) yes 

1 (Bi  ) no 

2 (B 2 ) yes 

3 (B  2 ) no 

Interpretation:  The  steady-state  non-switching  gain  exists;  it  stabil- 


izes configuration  0 and  2 (B^and  B^  )•  Since  the  probabilities  of 
being  in  configuration  1 and  3 (B  ^ and  B^  ) are  small  (tt^^  and  tt^) 
(unstable),  the  system  configuration  is  stabilized  during  the  non-switch- 
ing gain  G in  the  control  law 
’ —ns 

u ^ = G X ^ 

— t — ns  — t 
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From  the  results  in  Example  6.1,  the  design  configurations  are 


ranked  as  follows,  where  > is  defined  as  "is  better  than". 

> B^ 

1 k 

One  configuration  is  more  desirable  than  another  (B  > B ) if 


(6.2.9) 


3 . . ^ , 

^ TT^S  Tr?S  > 0 

^ 1-1  1- 1 


(negative  definite) 


(6.2.10) 


"i  k 

This  criterion  is  reasonable;  if  ^ ^ 5.  » then  the  expected  cost  using 

design  configuration  B-*  is  always  less  than  that  using  B . If  the  left 

hand  side  of  equation  (6.2.10)  is  not  negative  definite,  but  is  only 

semi-definite,  then  some  other  criterion  must  be  used  in  addition  to 

(6.2.10)  to  rank  the  various  designs.  For  example,  if  one  assumes  a 

uniform  distribution  of  the  initial  system  state  x^  in  the  unit  sphere, 

and  if  the  elements  of  the  diagonal  of  the  left  hand  side  of  equation 

(6.2.10)  are  all  non-positive,  then  the  trace  operator  may  be  used  as  a 

ranking  function.  If  the  trace  of  the  left  hand  side  of  equation  (6.2.10) 
*1  k 

is  negative,  then  > B • If  the  left  hand  side  of  equation  (6.2.10)  is 
not  semi -definite,  then  the  designer  must  choose  which  of  the  state 
variables  are  most  important  in  an  effort  to  eliminate  the  ambiguity  of 
equation  (6.2.10).  In  Example  6.1,  equation  (6.2.10)  alone  is  sufficient 
to  rank  the  designs. 

The  results  stated  in  (6.2.9)  are  somewhat  surprising.  First, 

consider  b^  and  • A control  input  at  time  t using  b^  enters  the 

T 

system  dynamics  in  state  x , where  x = [x  . x x ] . At  time  t+1, 

•3  ~ 

the  same  control  is  applied  to  state  x^^  with  a gain  of  .5;  also, 

X.,  ^ = x,  At  time  t+2,  that  control  is  again  applied  to  state  x, 

2,  t+1  3,t  t-r  2. 

with  a gain  of  .5  . Now,  consider 
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the  same  situation,  but  with  instead  of  b^  . In  this  case,  at  time 

tfl,  the  control  is  applied  to  state  x^,  with  a t)ain  of  .S,  but 

X,  = -x^  Therefore,  at  time  t+2,  the  neqative  value  of  the  oriijinal 

-*  § t • X f c 

control  is  applied  to  state  x^ , thus  partially  cancelling  the  effect  of 
the  original  input.  The  same  process  occurs  using  b^^  , but  is  delayed 
one  time  step;  thus,  the  control  affects  state  Xj^  positively  one  additional 
time  stop  when  b^  is  used.  Because  of  the  added  effectiveness  of  b^^ 
12.  14 

over  , B ^ li  » fact,  B ^ Tims,  even  after  accounting 

for  component  reliability,  configuration  B^,  which  has  no  comi)onent 

2 4 

redundancy  is  more  desirable  than  configuration  B or  B even  though 
4 

configuration  B employs  one  level  of  component  redundancy. 

Using  this  reasoning,  one  would  expect  B^  to  be  the  optimal  design 
choice;  however,  tl>e  example  demonstrates  that  this  is  not  the  case. 

From  G for  Case  iv) , note  that  the  control  which  is  applied  to  b , 
doi'onds  mostly  on  the  unstable  state  x^^,  wliile  more  emphasis  is  given 
to  states  x^  and  x^  in  the  calculation  of  the  control  for  actuator  b^  . 
Thus,  actuator  b^^  acts  partially  to  stabilize  the  dynamics  of  state  x^, 
while  actuator  b acts  partially  to  counteract  the  negative  effects  of 

tht'  subsystem  of  states  x.^  and  x^.  This  type  of  control  action  is  an 

example  of  the  use  of  functional  redundancy,  aiul  is  tic>t  possible  with 

j . 3 4 

design  configurations  B or  B . 

Tlu>  non-switching  gain  analysis  of  the  proposi'd  design  configura- 
tions yiehls  information  not  only  about  the  effect  of  various  actviator 

configurations  but  also  about  the  effect  of  compom-nt  reliability  oi; 

4 2 1 

tlie  exp«>ctt?d  p<'rformance.  Thus,  i.s  mort'  effective'  tlian  B , and  B 

I 4 t 

is  more  eeffoctive  than  B ; B atui  B are'  ve'rsiems  eif  tiu'  ce'uf  iiiur.it  ieins 


B and  B , respectively,  with  one  level  of  component  redundancy.  Con- 
figuration B^  is  an  example  of  functional  redundancy;  both  actuators 
provide  control  input  to  the  s^une  system,  but  are  not  identical  components. 
Thus,  the  additional  reliability  of  component  redundancy  contributes 
to  ranking  (6.2.9).  The  trade-off  between  system  performance  and  system 
reliability  will  be  further  demonstrated  in  Section  3. 
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6.3  A Trade-Off  of  System  Performance  Versus  Reliabilitv 


The  non-switching  gain  methodology  can  be  used  to  study  the 

relative  effects  of  actuator  reliability  and  actuator  effectiveness 

on  expected  system  performance.  If  a designer  has  a choice  between 

using  a high  reliability  actuator  rather  than  one  with  relatively  low 

reliability,  but  with  a higher  effectiveness,  on  what  basis  can  a 

decision  be  made?  In  Example  6.2,  two  actuators  are  considered.  Each 

actuator  may  fail  to  an  actuator  of  gain  zero  (0)  and  be  repaired 

(replaced) . The  probabilities  of  failure  and  repair  are  p^  and  p^  , 

i i 

where  i=0  or  1 and  refers  to  the  actuator  (b^  or  b^  , respectively). 

One  actuator  (b^  ) has  good  reliability,  but  the  actuator  gain  is  unity. 
A second  actuator  (b^^  ) has  an  actuator  gain  of  ten  (higher  effective- 
ness), and  a lower  reliability.  If  the  actuators  had  the  same  relia- 
bility, then  actuator  b would  be  preferable — it  incurs  a smaller  cost 
for  the  same  effect.  In  Case  i)  of  Example  6.2,  this  reasoning  is 
demonstrated  numerically;  the  steady-state  non-switching  gain  favors 
actuator  b^  (the  second  column  of  ).  (The  two  rows  of  the  gain 
matrix  are  compared;  the  top  row  corresponds  to  actuator  b^.) 

In  Cases  ii)  and  iii)  of  Example  6.2,  the  reliability  of  actuator 
b^  is  lower  than  the  reliability  of  actuator  . In  Case  ii)  the 
probability  of  failure  per  unit  time  of  actuator  b^  is  five  times 
greater  than  the  probability  of  failiure  per  unit  time  of  actuator  b^  ; 
in  Case  iii) , it  is  ten  times  greater.  The  probabilities  of  repair  per 
unit  time  for  actuator  b are  also  lower  than  for  actuator  b^  . 
Therefore,  actuator  b is  significantly  less  reliable  than  actuator  b^  . 
Note  that  in  Case  ii),  the  optimal  non-switching  steady-state  controller 


favors  actuator  b by  a gain  factor  of  2.5  - 2.6;  in  Case  i),  actuator 
— 0 

bj^  is  favored  by  a gain  factor  of  2.3.  In  Case  iii)  , actuator  b^  is 
favored  by  a gain  factor  of  5.1.  Tnus,  the  non-switching  gain  calcula- 
tions can  be  quite  sensitive  to  changes  in  component  reliability. 

Although  the  configuration  states  are  identical  for  all  three  Cases  of 
Example  6.2,  the  configuration  dynamics  are  modified  by  the  changes  in 
actuator  reliability.  The  effect  of  modifications  in  actuator  reliability 
on  the  non-switching  steady-state  gain  and  cost  is  pronounced.  The 
steady-state  gain  is  very  sensitive  to  the  actuator  reliabilities;  the 
expected  steady-state  cost  increases  as  the  reliability  decreases.  A 


second  effect  demonstrated  by  Example  6.2  is  interesting.  In  Case  i) , 


configuration  state  2 is  not  stabilized  by  the  non-switching  gain.  As 
the  reliability  of  actuator  b^  decreases,  the  average  steady-state 
probability  that  the  configuration  is  state  2 (actuator  b^  failed, 
actuator  b^  operational)  increases.  Therefore,  the  non-switching  gain 
solution  must  concentrate  more  effort  on  stabilizing  configuration  state 


2.  Note  that  in  Cases  ii)  and  iii) , configuration  state  2 is  stabilized 
by  the  non-switching  gain  solution.  It  is  interesting  to  note  also  that 
the  non-switching  gains  in  Cases  ii)  and  iii)  are  robust  with  respect  to 
configuration  states  0,  1 and  2.  (Configuration  state  3 is  uncontrolla- 


ble.) 


The  system  dynamics  in  Example  6.2  are 


=Ax^+B,,^,u 
-t+1 1 -k(t)  -t 


k(t)  e I 


(6.3.1) 


(6.3.2) 


where  I * {o,l,2,3}  and  x = lx  x x 1 The  set  { B . 

t ii"“0 

of  configuration  states  is  given  in  Example  6.2.  The  cost  to  be 


Expected  cost  = x C x 
Stability; 

Configuration  Stable 

0 (Bg  ) yes 

1 (B  ) yes 

2 (B 2 ) no 

3 (B  2 ) 

Interpretations:  The  system  x = [A  + B.G  ] x is  stable  only  for 

— t+i  — — 1 — ns  — t 

i=0  and  1.  The  probcibilities  of  the  configuration  being  in  states  2 and  3 
(ir^  and  ir^)  are  small;  the  system  configuration  is  stabilized  using  the 
control  gain  G in  the  control  law 
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Expected  cost  = 21  £.  ii 
Stability: 


Configuration 


0 (Bq) 


1 (B  J ) 


2 (B2  ) 

3 (B3  ) 


Interpretation:  The  system 


=[A  + B.G  ]x^ 
— 1+1  — — 1— ns  — t 
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Stable 

yes 

yes 

yes 


no 


is  stable  for  i = 0,1,2, 

Configuration  state  2 is  stabilized  because  the  probability  of  the 


configuration  state  being  2 (B^  ) is  larger  than  in  Case  i) . 


Example  6.2  Case  ill) 


P-  = .01  p = .98 


p = .10  p = .90  n 

1 1 


.8909 

.009172 

.09891 

.001010 


Convergent : 
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Expected  cost  = x C x 
Stability: 


Configuration 


0 (Bq) 


1 (B^  ) 


2 (B^  ) 


Stable 

yes 

yes 

yes 


3 (B^  ) 


no 


Interpretation:  The  system 


X . . , = [A  + B . G ] X 

— 1+1  — — 1— ns  — t 


is  stcible  for  i = 0,1,2. 

Configuration  state  2 is  stabilized  because  the  probability  of  the 


configuration  state  being  2 (5^2  ^ larger  than  in  Case  i) . 
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6.4  Suimnary. 

In  this  Chapter,  two  applications  of  the  non-switching  gain  method- 
ology to  computer-aided  design  (CAD)  were  presented.  The  purpose  of 
these  examples  was  to  demonstrate  the  usefulness  of  the  non-switching 
gain  methodology  in  the  design  process.  CAD  has  two  uses:  First,  it  is 
used  by  the  system  designer  in  the  evaluation  and  design  of  a system. 
Second,  it  is  quite  useful  to  the  theorist.  In  this  research,  for 
excunple,  without  CAD  techniques,  a thorough  knowledge  of  the  methodologies 
presented  in  this  report  could  not  have  been  gained.  The  equations 
describing  the  switching  and  non-switching  gain  methodologies  can  be 
derived,  but  their  meaning  in  a specific  context  cannot  be  determined 
theoretically.  The  purpose  of  this  research  was  to  study  the  inter- 
actions between  system  reliability  and  optimal  control.  The  method- 
ologies presented  in  this  report  allow  this  study  to  proceed.  The  two 
Examples  of  this  Chapter  study  two  specific  areas  of  interaction 
between  system  reliability  and  control.  The  door  has  now  been  opened  to 
the  answers  to  questions  on  reliable  control  system  designs.  Computer- 
aided  design  can  provide  the  signposts  to  these  answers. 


174 


CHAPTER  7 
CRITIQUE 


7 . 1 Introduction. 

In  this  Chapter,  the  major  results  of  the  report  will  be  summarized. 
In  Chapters  3 and  4,  the  switching  gain  solution  was  developed  and 
extended  suboptimally  to  stochastic  systems.  In  Chapter  5,  the  non- 
switching gain  solution  was  developed.  The  problems  associated  with 
system  stability,  including  definitions  of  what  constitutes  a stable 
system,  and  with  the  steady-state  solutions  to  Problems  A (Sections  3 
through  5)  and  B (Section  6)  were  studied  in  detail  in  Section  7.  The 
equivalence  of  the  two  approaches  to  the  non-switching  gain  solution  is 
proved  in  Section  8.  The  existence  of  a robust  steady-state  linear 
feedback  control  system  was  studied  in  Section  9. 

In  the  following  sections,  each  major  result  will  be  discussed;  in 
Section  5,  some  suggestions  for  future  directions  in  research  will 
be  made. 

7 . 2 The  Switching  Gain  Solution. 

The  switching  gain  solution  was  derived  in  Chapter  3 as  a control 
methodology  for  linear  system  with  quadratic  cost  criteria  and  variable 
actuator  configurations.  The  resulting  control  law  was  to  account  for 
the  failure,  repair  and  reconfiguration  of  the  actuators  by  switching 
the  control  gain  on  detection  of  a change  in  configuration.  This  type 
of  control  law  is,  from  Chapter  1,  Section  4,  a class  II  reliable  control 
methodology;  an  active  (switching)  controller  is  used  with  a passive 


oont  iyiiiat  ion  dosiqn. 


i7i>  : 

7.2.1  Dot ^'rmini^5tic  Optimal  Solution.  | 

Tho  swilchinq  yain  solutiot)  Ohaptor  2 is  ilorivod  as  tht'  optimal  j 

solut  ion  for  th»'  iliscri't«'-t  imo  ilotorminist  ic  optimal  control  problem. 

Tt  is  the  optimal  control  simply  bt'c.tusc  tlio  structure  <.>l'  tlie  iliscrete- 
t ime  system  allows  perfect  observations  of  tire  system  structure  witli 
orre-step  delay.  Thruefore,  Mit're  ir.  no  need  for  fbt'  control  law  to 
liave  a dual  effect;  in  fact,  tliere  cart  be  no  dual  r'ffi'ct,  since  tlie 
control  law  does  not  affect  the  obsi'ivation  process,  for  almost  all 
values  of  the  control. 

A minor  drawback  to  the  switclriny  yain  solution  is  the  computa- 
tional burdr'ti  of  iteratinq  tiro  Riccat  i-likr'  equations  ( i..l.()),  anil  solvinq 
for  the  optimal  control  usinq  equation  (3.J.7),  backward  in  time  for 
each  timi'  instant  of  tin'  control  interval,  or  until  the  .steady-state 
solution  is  achieved,  when  one  exists.  Kortunately  this  computation  is 
done  off-line,  .rnd  the  various  optimal  qaiirs  are  then  stored  fi>r  on-line 
vtse.  On-lirre,  the  controller  simply  determines  which  structure  the 
system  was  in  at  the  jrrevious  time  instant  and  chooses  the  correspondinq 
(stored)  qain.  Tl>e  control  law  is  then  a inear  feedback  control  usinq 
th.it  particular  qain. 

Non-Kxtendabi  I ity  to  Stoclnistic  Systems. 

' fi  t ■ t nn.if  e I y , the  sw»  t ch  i nq-qa  i n solution  does  not  extend  optimally 
'■m  wii.tr  HOI-.'  I-,  pti'-ient.  Wht'n  noise  is  present,  it  is  no 
1.  t..  I ill  to  letetmine  ex.ict  ly  the  previous  value 
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in  such  a case,  the  optimal  control  law  exhibits  a dual  effect;  i.e., 
the  control  law  influences  the  measurement  of  the  system  structure.  In 
a real-life  situation,  it  is  unlikely  that  a system  witli  no  internal 
noise  will  be  found.  Unfortunately,  the  optimal  (dual)  control  law  is, 
in  practice,  unsolvable  due  to  the  immense  computer  resources  whicli  are 
required. 

7.2.3  Suboptimal  Extensions. 

Because  of  the  dual  control  effect,  the  deterministic  optimal 
solution  is  the  only  closed-form  solution  available.  Thus,  it  is  in 
our  interest  to  look  for  suboptimal  methodologies  wliicli  extend  the 
switching  gain  solution  to  the  stochastic  case.  In  Chapter  4,  two  of 
these  methodologies  were  studied;  Hypothesis  testing  and  dual  identi- 
fication, While  hypothesis  testing  is  a measurement  strategy,  dual 
identification  modifies  the  control  in  order  to  guarantee  a perfect 
observation  of  the  system  structure  with  the  next  measurement.  Both 
methodologies  are  presented  in  their  simplest  form,  since  the  problems 
of  stochastic  control  of  systems  with  variable  structure  are  not  within 
the  scope  of  this  research.  Two  comments  are  in  order,  liowever: 

First,  at  least  in  the  form  presented  in  Chapter  4,  a dual  identifica- 
tion algoritlun  is  computationally  intensive.  Since  it  is  an  on-line 
algorithm,  a significant  computational  capacity  may  be  required  in  its 
implementation.  Second,  it  is  observed  that  the  optimal  stochastic 
control  law,  if  it  could  be  calculated,  would  rely  on  botli  estimation 
and  dual  control,  the  two  concepts  wl\ich  are  represented  in  Chapter  4 by 


hypotliesis  testing  and  dual  identification,  respectively. 
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In  a suboptimal  implementation  using  dual  identification,  the 
algorithm  would  most  likely  be  used  only  at  intervals;  the  implementa- 
tion would  rely  on  an  estimation  algor itJim  for  the  remainder  of  the 
time.  This  scheme  would  attempt  to  minimize  the  degrading  effect  of 
dual  identification  on  the  state  trajectory  by  using  it  only  to  guarantee 
that  the  estimation  algorithm  was  tracking  the  system  configuration 
properly.  Thus,  the  system  response  would  be  roughly  periodic,  with 
the  state  being  driven  away  from  the  origin  in  order  to  obtain  an 
accurate  estimate  of  the  configuration,  and  decaying  back  toward  zero 
between  uses  of  the  dual  identification  algorithm. 

This  type  of  control  strategy  deserves  some  attention  in  future 
research  activities.  It  is  similar  to  the  class  of  self-testing 
systems  which  perform  diagnostic  testing  of  their  configurations 
at  intervals.  It  is  also,  at  present,  the  only  methodology  which  takes 
advantage  of  the  dual  property  of  the  control  law  in  systems  with 
variable,  imperfectly  observed,  structure. 
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7 . 3 The  Non-Switching  Gain  Solution. 

The  non-switching  gain  solution  of  Chapter  5 was  derived  as  an 
alternative  to  the  switching  gain  solution  of  Chapter  3.  Although 
the  non-switching  solution  is,  in  general,  suboptimal,  the  on-line 
complexity  of  the  solution  is  less  demanding  than  that  of  the  switching 
gain  solution.  On-line,  the  non-switching  gain  solution  has  the  same 
complexity  as  does  the  standard  linear  quadratic  solution.  Off-line, 
the  computational  requirements  are  equivalent  to  those  of  the  switching 
gain  solution. 


7.3.1  The  Necessary  Conditions — Unsolvability. 

When  the  non-switching  control  problem  is  formulated  as  an 


equivalent  deterministic  control  problem  (Chapter  5,  Section  4) , the 
necessary  conditions  from  the  matrix  minimum  principle  [Athans,41] 
yield  a two-point  boundary  value  problem  which  is  not  explicitly 


solvable;  at  the  present  time,  the  solution  to  this  problem  appears 
intractable.  The  necessary  conditions  are  used,  however,  in  conjunction 
with  an  equivalent  problem  (Chapter  5,  Section  6),  to  prove  some  strong 
properties  of  the  solution  to  the  equivalent  problem. 


7.3.2  The  Equivalent  Problem. 

The  equivalent  problem  formulated  in  Section  6 of  Chapter  5 lias 


the  advantage  over  the  original  formulation  that  a closed-form  expression 
for  the  solution  can  bo  readily  obtained.  From  the  necessary  conditions 
of  Section  5 in  Chapter  5 for  the  original  formulation,  it  is  shown  tliat 
the  accumulated  costs  over  the  control  interval  for  a specified  gain 


sequence  are  identical  for  the  two  formulations.  From  this,  in  Section  8 
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of  Chapter  5,  it  is  shown  that  if  the  steady-state  solutions  to  both 
problems  exist,  then  they  are  identical.  This  is  a major  result,  since 
the  steady-state  solution  to  the  second  formulation  is  calculable, 
while  the  solution  to  the  first  formulation  is  not. 

7.3.3  Existence  of  a Stabilizing  Gain. 

Only  one  major  result  remains;  one  would  hope  that  the  steady-state 
solution  to  the  second  formulation  exists  if  and  only  is  the  steady-state 
solution  to  the  first  formulation  exists.  In  Section  7 of  Chapter  5,  the 
meaning  of  "steady-state"  is  precisely  defined  for  both  problems.  In 
order  for  the  concept  of  a steady-state  solution  to  be  well-defined,  an 
exact  definition  of  stability  must  be  given.  Two  definitions  are  present- 
ed. Stability  is  defined  as  the  usual  concept  of  mean-square  stability. 

A definition  of  cost- stability  is  presented  as  the  condition  when  the 
expected  cost  for  the  infinite  horizon  problem  (unnormalized  by  time) 
is  bounded.  It  is  proved  that  the  solutions  to  the  two  formulations 
are  equivalent  in  that  one  solution  is  cost-stabilizing  if  and  only  if 
the  other  is  also.  Cost  stability  is  shown  to  imply  mean-square 
stability;  the  reverse  is  not  necessarily  true. 

7.3.4  Problems  with  Convergence. 

There  are  two  criticisms  of  the  results  of  Chapter  5.  First, 
although  cost-stability  is  not  implied  by  mean-square  stability,  it  is 
possible  that,  for  the  specific  form  of  the  non-switching  ga^n  solution, 
the  two  definitions  are  equivalent.  This  is  a minor  point,  in  that  the 
equivalence  result  is  already  very  strong;  it  yields  a procedure  for 
the  calculation  of  the  steady-state  solution  to  the  two  point  boundary 
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value  problem  which  converges  if  and  only  if  that  solution  exists. 

Second,  there  is  still  a minor  problem  concerning  the  convergence 
of  the  non-switching  gain  solution.  The  equivalence  theorems  of 
Chapter  5 only  require  the  solution  to  have  a steady-state,  which  may 
be  a limit  cycle.  A limit  cycle  is  still  copacetic,  but  it  is  harder 
to  implement  than  one  gain  would  be.  Therefore,  it  is  desired  that 
conditions  be  found  for  which  the  possibility  of  a limit  cycle  is 
ruled  out. 

Thus,  two  possible  topics  for  future  research  are  the  examination 
of  the  exact  relationship  between  cost-stability  and  mean-square  stability 
for  the  non-switching  solution  and  the  determination  of  conditions  for 
which  the  possibility  of  limit  cycles  as  solutions  is  eliminated. 

7.3.5  Existence  of  a Robust  Gain. 

A spin-off  of  the  non-switching  gain  solution  of  Chapter  5 is 
the  development  of  an  algoritlim  which  determines  when  a robust  gain 
for  a set  of  linear  systems  exists  (Section  9).  A robust  gain  is  a 
gain  wliich  stabilizes  each  mode  of  the  system  configuration  regard- 
less of  the  configuration  dynamics.  This  algoritlim  is  developed  by 
noting  that  the  robustness  problem  can  be  reformulated  as  a non-switch- 
ing gain  problem.  Since  the  non-switching  gain  is,  in  the  steady-state 
case,  the  solution  to  the  first  formulation  (Section  4,  Chai'ter  5),  and 
since  it  is  stabilizing  if  and  only  if  a stabilizing  gain  exists,  then 
by  the  special  structure  of  the  robust  formulation  (Section  9) , the 
steady-state  non-switching  gain  is  robust  when  it  exists.  In  addition, 
if  the  non-switching  solution  is  not  cost-stabilizing,  then  no  robust 
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7.4  Computer-Aided  Design. 

Chapter  6 demt  nstrates  the  usefulness  of  the  non-switching 
gain  calculations  in  computer-aided  design  (CAD) . These  calculations 
provide  the  backbone  for  comparison  studies  on  the  relative  system 
effectiveness  of  various  designs.  In  the  first  example,  it  is  demon- 
strated that  the  non-switching  control  methodology  yields  a numerical 
value  based  on  the  expected  performance  of  a design  configuration 
over  the  effect  of  the  structural  dynamics.  This  example  demonstrates 
that  relatively  subtle  qualities  of  an  actuator  can  be  used  to  rank 
various  actuator  configurations;  in  this  case,  the  ranking  depends 
on  the  manner  in  which  the  control  affected  the  system  state  and  is 
not  obvious  on  a casual  inspection  of  the  configuration. 

The  second  example  demonstrates  the  ability  of  the  non-switching 
gain  methodology  to  observe  the  trade-off  between  high  reliability  and 
high  effectiveness  in  an  actuator.  Both  qualities  are  desirable,  but 
in  this  example,  one  actuator  is  highly  reliable,  while  the  second 
actuator  is  not  as  reliable,  but  is  highly  effective  in  that  it  incurs 
a much  smaller  cost  in  applying  the  same  control  effect  to  tl\e  system. 

The  non-switching  gain  problem  is  solved  for  a range  of  actuator  reli- 
abilities for  the  highly  effective  sensor.  It  is  demonstrated  tliat 
the  trend  exists  to  depend  more  heavily  on  the  high  reliability  sensor 
as  the  reliability  of  the  highly  effective  sensor  decreases,  even 
though  the  operation  of  the  highly  reliable  sensor  incurs  more  cost. 

Chapter  6 only  touches  upon  the  field  of  computer-aided  design. 

There  is  much  work  to  be  done  in  this  field,  and  the  purpose  of  Chapter  6 
is  only  to  establish  the  usefulness  of  the  non-switching  gain  metluxiology 


in  the  design  process.  In  the  future,  the  applicability  of  the  non 


switching  gain  methodology  to  CAD  should  be  studied  in  great  detail 


in  particular,  a comprehensive  methodology  for  the  application  of  the 


techniques  of  Chapter  5 to  CAD  should  be  developed.  This  methodology 


should  include  a strong  argument  for  the  validity  of  using  the  non-switch 


ing  methodology  in  CAD.  Specifically,  research  needs  to  be  carried  out 


on  the  relationship  of  the  costs  incurred  by  various  design  configurations 


this  is  similar  to  justifying  the  use  of  the  quadratic  cost  criterion 


in  the  linear  quadratic  regulator.  In  order  to  compare  two  designs,  a 


valid  basis  of  comparison,  or  cost  index,  must  exist.  The  non-switching 


methodology  is  proposed  as  being  a valid  cost  index  for  the  class  of 


systems  for  which  it  is  applicable;  this  conjecture  should  be  verified 


In  addition  to  the  usefulness  of  the  non-switching  methodology,  it 


has  been  mentioned  previously  that  a valid  definition  for  a reliable 


design  is  that  the  design  is  cost-stabilizable.  Since,  for  the  deter 


ministic  control  problem  presented  in  Chapter  3,  the  switching  gain 


solution  is  the  optimal  solution,  the  existence  of  the  steady-state 


switching  gain  solution  is  equivalent  to  the  stabilizability  of  that 


design.  Hence,  the  existence  of  the  steady-state  switching  gain  solution 


is  necessary  and  sufficient  to  classify  a design  reliable 


In  theory,  the  computation  of  the  steady-state  switching  gain 


solution  can  be  used  as  a method  in  CAD  for  determining  if  a proposed 


in  a stochastic  environment 


therefore,  the  switching  gain  solution  is  not  an  absolute  measure  of  the 


stabilizability  of  the  design.  In  the  future,  research  should  be 
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concentrated  on  the  development  of  the  concept  of  stabilizability  to 
more  general  stochastic  systems  than  has  been  done  previously.  An 
example  of  work  in  this  direction  has  been  given  with  the  Uncertainty 
Threshold  Principle  [Athans,  cQ. , 37] , which  is  basically  the  deter- 

mination of  conditions  of  stabilizability  for  a specific  system  with  a 
specific  type  of  control  law.  The  work  on  the  existence  of  the  non- 
switching gain  solution  for  a simple  system  (Chapter  2,  Section  7) 
is  another  example.  It  has  been  demonstrated  in  this  research  that  the 
concepts  of  systems  reliability  and  stabilizability  are  crucially 
interconnected.  It  is  left  to  future  research  to  determine  more  general 
conditions  of  reliability  and  stabilizability  and  to  implement  these 
conditions  in  computer  algorithms  which  can  be  used  by  the  designer. 
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7.5  Suggestions  for  Future  Research. 

Several  suggestions  for  future  research  have  been  presented  in 
Sections  2,3  and  4 of  this  Chapter.  In  this  Section,  a summary  of  these 
suggestions  will  be  given. 

In  Chapter  1,  three  classes  of  reliable  control  methodologies 
were  given.  These  are 

I)  Passive  (Robust)  Controller  Design 

II)  Active  (Switching)  Controller,  Passive  Configuration 
Design 

III)  Active  Controller,  Active  Configuration  Design 

Of  the  methodologies  presented  in  this  report,  the  non-switching 
gain  design  is  a class  I methodology,  and  the  switching  gain  design  is 
a class  II  methodology.  Class  III  methodologies  are  not  represented 
in  this  report.  This  class  is  currently  largely  in  the  realm  of 
"blue  sky"  theory.  Unfortunately,  there  is  as  yet  no  adequate  model 
of  configuration  dynamics  which  exhibits  a state  and  control  structure. 
Over  the  next  ten  years,  one  should  see  much  research  activity  in  the 
area  of  class  III  methodologies  and  their  control  structures. 

In  class  II  methodologies,  much  effort  should  be  concentrated  on 
extensions,  either  optimal  or  suboptiroal,  of  the  switching  class  of 
control  laws  to  stochastic  systems.  At  present,  most  work  has  been  done 
in  estimation  theory,  since  the  difficulties  associated  with  dual 
control  are  widely  recognized.  The  ability  of  a control  law  to  perform 
diagnostic  testing  for  changes  in  configuration  has  yet  to  be  exploited 
theoretically,  although  many  heuristic  algorithms  have  been  used,  both 
in  control  systems  and  in  the  more  established  field  of  fault  detection 


t 


I 


I 
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and  identification  in  digital  systems.  Dual  control  is  a form  of  self- 
testing, and  can  be  utilized  as  such,  even  if  an  optimal  control  is 
not  known.  The  dual  identification  methodology  of  Chapter  4 is  an 
example.  This  field  requires  a large  effort,  and  should  be  rich  in 
research  opportunities. 

The  class  I methodologies  are  represented  in  this  research  by  the 
non-switching  gain  solution.  The  work  done  in  Chapter  5 on  mean-square 
stability  and  cost-stability  of  solutions  is  not  unique  to  this  class  of 
problems.  Much  remains  to  be  done  in  the  classification  of  what  consti- 
tutes a stabilizable  system,  whether  with  respect  to  a non-switching 
control  law  or  something  more  general. 

Since  reliability  can  be  defined  as  stabilizability  with  respect 
to  some  class  of  control  laws,  research  into  the  stabilizability  of 
dynamic  configuration  systems  is  the  key  issue  in  reliable  control 
system  designs.  Much  work,  including  this  research,  has  been  done  on 
the  assumption  that  the  system  is  stabilizable;  however,  little  progress 
has  been  made  in  determining  why  a given  design  is  stabilizable. 

Although  iterative  tests  were  developed  in  this  report  for  determining 
stabilizcibili ty , a thorough  understanding  of  the  reason  these  tests 
either  converge  or  fail  to  converge  is  lacking.  Much  work  still  must  be 
done.  With  this  should  come  a resolution  of  the  problems  with  limit 
cycle  steady-state  solutions  to  the  non-switching  gain  methodology. 

In  Chapter  6,  the  usefulness  of  the  non-switching  gain  solution  in 
computer-aided  design  was  demonstrated.  CAD  is  a field  unto  itself;  many 
opportunities  exist  for  research  in  this  area.  Unfortunately,  most 


research  is  application-specific.  CAD  is  useful  not  only  to  tlie  designer. 
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but  also  to  the  researcher.  It  is  a powerful  tool  in  the  building  of 
the  concepts  of  reliable  control  systems  design,  and  it  should  be 
developed  in  parallel  with  any  future  research. 


7 . 6 Summary. 

I In  summary,  the  main  purpose  of  this  research  was  to  establish  a 

foundation  in  reliable  control  system  design  methodology  which  would 

I 

r provide  the  basic  concept  of  a reliable  control  system.  In  achieving 

this  goal,  the  linear  quadratic  variable  actuator  control  problem  was 
studied  in  some  detail.  Optimization  problems  were  formulated  which 
represented  both  system  performance  (in  the  quadratic  performance  index) 
and  system  reliability  (in  the  expectation  of  the  performance  index  over 
all  possible  structural  trajectories) . The  optimal  control  law  was 

I 

solved  analytically  for  the  deterministic  system;  this  was  the  switching 
gain  solution.  It  was  clearly  illustrated  by  example  in  Chapter  2 that 
the  switching  gain  control  law  could  not  be  extended  analytically  to 
the  control  of  stochastic  systems.  This  example  demonstrated  the  dual 
effect  of  the  control  law;  in  general,  the  control  law  will  influence 
the  measurement  accuracy  optimally  (in  the  sense  of  minimizing  expected 
cost)  when  the  control  can  influence  the  accuracy. 

Stochastic  extensions  to  the  switching  gain  methodology  were  proposed 
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In  Chapter  5,  the  non-switching  gain  solution  was  developed.  This 
solution  led  to  an  algorithm  for  the  determination  of  robust  linear 
constant  gain  control  laws  for  a set  of  linear  systems  with  different 
actuator  configurations.  In  addition,  the  resulting  gains  are  optimal 
with  respect  to  a given  quadratic  performance  index  and  exist  if  and 
only  if  any  robust  gain  exists. 

In  conclusion,  the  unifying  concept  of  this  report  is:  What 
constitutes  a reliable  control  system,  or  a reliable  design?  A major 
connection  was  established  in  this  research  between  the  concepts  of 
reliability  and  stabilizability . Iterative  procedures  were  developed 
for  the  determination  of  whether  or  not  a given  linear  system  of  the 
type  considered  in  this  report  is  reliable,  with  respect  to  both  class 
I and  class  II  controllers;  i.e.,  non-switching  and  switching  gain 
controllers,  respectively. 


■J 
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DEFINITIONS  FROM  MIL-STD-721B 
25  August  1966 


RELIABILITY 

The  probability  that  an  item  will  perform  its  intended  function 
for  a specified  interval  under  stated  conditions. 

AVAILABILITY 

A measure  of  the  degree  to  which  an  item  is  in  the  operable  and 
committable  state  at  the  start  of  the  mission,  when  the  mission  is 
called  for  at  an  unknown  (random)  point  in  time. 

DEPENDABILITY 

A measure  of  the  item  operating  condition  at  one  or  more  points 
during  the  mission,  including  the  effects  of  Reliability,  Maintain 
ability  and  Survivcibility,  given  the  item  condition (s)  at  the  start 
of  the  mission.  It  may  be  stated  as  the  probability  that  an  item  will 
(a)  enter  or  occupy  any  one  of  its  required  operational  modes  during  a 
specific  mission,  (b)  perform  the  functions  associated  with  those 
operational  modes. 

CAPABILITY 

A measure  of  the  ability  of  an  item  to  achieve  mission  objec- 
tives given  the  conditions  during  the  mission. 

OPERABLE 

The  state  of  being  able  to  perform  the  intended  function. 
MAINTAINABILITY 

A characteristic  of  design  and  installation  which  is  expressed 
as  the  probability  that  an  item  will  be  retained  in  or  restored  to  a 
specific  condition  within  a given  pej.iod  of  time,  when  the  main- 
tenance is  performed  in  accordance  with  prescribed  procedures  and 
resources. 

SURVIVABILITY 

The  measure  of  the  degree  to  which  an  item  will  withstand  hostile 
man-made  environment  and  not  suffer  abortive  impairment  of  its 
eUaility  to  accomplish  its  designated  mission. 


I: 
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A2 . 1 Exact  Optimal  Solution  for  Deterministic  Case,  Chapter  2 , 

Section  2. 

From  (2.2.7)  and  using  dynamic  prograunming,  we  wish  to  minimize 

2 2 

V(x^,  k(t-l),  u^,  t)  = E(qx^  + ru^ 

+ V (a  x^+b  (t)  I • 1 • 1) 

* 

where  V (‘.kCt),  t+1)  represents  the  minimum  cost-to-go,  given 
)t(t)  at  time  t+1. 

This  minimization  can  be  carried  out  because  x^  is  known  exactly 

at  time  t,  and  therefore  ^^is  knownexactly  by  equation  (2.2.10). 

The  control  u^  is  computed  from 

0 = I qx^  + ru^  + TT-  V (ax.  + bu  , k=0,t+l) 

3u  y t t 0^  t t 

+ TT,  V (a  X.+  ^u.  , k=l,t+l)  ) (A2.1.2) 

and  the  assumption  that 

V (x^ , k=i , t)  = x^  S . (A2 .1.3) 

t t 1 # t 

resulting  in  equation  (2.2.8).  Equations  (2.2.12)  and  (2.2.13)  are 
then  obtained  by  substitution  of  (2.2.8)  into  (A2.1.1);  these 
equations  validate  assumption  (A2.1.3)  by  induction. 


-i.y  -w. . 
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A2 . 2 Exact  Optimal  Solution  for  Stochastic  Case , T=0.  1 , 2 = 

(1-d  example). 

The  formulation  is  the  same  as  in  A2.1,  except  the  system  is 
now  represented  by 

^tl  = ‘^k(t)  “t  ^ ^t  (K2.2.X) 

is  white  noise  with  zero  mean,  variance  E,  and  probability  dis- 
tribution p (C) , which  is  uncorrelated  with  any  other  variable.  To 
illustrate  the  complexity  of  the  solution,  the  time  set  is  chosen  as 

r 1 * * 

1 0,1, 2).  The  problem  is  to  find  u and  u such  that 

r . 0 i 


V(Xq,0)  = E(J)  = E 


■ U 1 

g (x2q+  u^r)  +x2q|xQ,TLo 


(A2.2.2) 


is  minimized.  Let  V denote  the  minimum  value  of  V.  Assume 

Ut=4>t{Zt>  (A2.2.3) 

where  is  a mapping  from  the  information  at  time  t (Z  ) into  the 
t ^ 


control  space. 


\ *0'  “o “t-1  ' *t^ 


then 


min  Elx^q  + u^r  + v‘(x,,l)|  Zq! 

">o“o’ 

by  dyneunic  programming.  Also 


V (x-,0) 


(A2.2.4) 


(A2.2.5) 


V*(Xj^,l)  «=  min  e!  x^  q + u^  r +V  2^^  j 


• 2 

But  V (x.,2)  = x_q,  so  (A2.2.6)  becomes 


(A2.2.6) 


(A2.2.7) 


* ( 2 2 2,1 

V (X  ,1)  =*  min  E Xj^q+  r + X2q|Zj^ 

Ui=</>i(Zi)  ' 

^ 


I ^1'  1' 


{A2.2.8) 


I 


now,  * tig 


(A2.2.8)  = 


(22 

min  < x.q  + u.r 

Uj=<t)i(Zi)  [ 

I ^ ‘"iV  I 


(A2.2.9) 


where  Tr^djl)  is  the  probability  that  - i,  given  Z^.  Bringing  the 


expectation  inside  the  sum, 


(22 

!.2.9)  = min  < x q + u r 

Ui=4>i(Zi)  ( 

+ (l|  1)  (a^x^  + + 5 + 2ab^x^Uj^)q| 


(A2.2.10) 


Differentiating  (A2.2.10)  w.r.t.  and  setting  the  result  equal  to 


0 » 2ru 


^ ^ TT.  (l|l)  {2b^Uj^  + 2ab^Xj^)q 


{A2.2.11) 


r + ^ TT^(l|l)b^  q 


(A2.2.12) 


Substituting  (A2.2.12)  back  into  (A2.2.10),  define  and  as 


• =q 


= (a  + l)q 


(A2.2,13) 


2 li^ 


TTidlDbiJ  q 


(A2.2.14) 


n 


V (Xj,l) 


X^Si  . 


(A2.2.15) 


A few  remarks  must  be  made  about  the  probability  distribution  over  k^, 
given  or 

Notation; 

Tr^(t|t)  = probability  that  k^  = i , given  the  available  information  Z^. 

Ti^(t|t+1)  = probability  that  k^  = i , given  the  available  information 

^t+1* 


From  the  Markov  property, 
1T(t(t)  = PTT(t-l|t) 


(A2.2.16) 


Equation  (A2.2.16)  is  the  propagation  equation  for  the  distribution  U. 


The  form  of  the  update  equation  is  given  and  proved  in  the  following 
lemma: 


Lemma  a2 . 1 : 


(tlt+1) 


(A2.2.17) 


Proof ; 


Note  that 


p<vr“Wt’  ' p(x^+ilz^.u^,k(t)=i) 

where  u^  is  not  a random  variable.  Also, 
7T^(t|t)  » p(k(t)-i|z^) 

p(k(t)“i,TiQ  ’‘t^ 

P<2.0  


I 


It 
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then  {A2.2.17)  becomes: 


p(k(t)=ilz^.^^)  = 

which  is  Bayes  rule. 


p(Xt^llZt»Ut,k(t)=i)p(k(t)=ilZt) 


P(Xt,i|Zt'^> 


Q.E.D. 


Returning  to  equation  (A2.2.5),  and  substituting  (A2.2.1S), 

E + u^r  + x^S  + tJZqI  (A2.2.18) 


V (x  ,0)  = min 

u =(|>  (Z  ) 

0 ^0  V o 


2 2 

= min  E i x^q  + u^r  + nq 


+ X, 


q(l+a^)  - 


2 2 
q a 


r + 


(A2.2,19) 


min 


XQq  t u^r  t cq 


J 


F 


n 
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Equation  (A2.2.21)  is  a ccMnbination  of  equations  (A2.2.16)  and  (A2.2.17). 
Equation  (A2.2.20)  can  only  be  solved  numerically  (in  general):  this 
requires  a numerical  minimization  of  a function  the  computation  of  which 
requires  four  numerical  integrations  — a difficult  tas)(. 
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A2.3  Exact  Solution  of  Stochast ic  Case  Over  T » 0,  1,  2 = 

Specific  Koim  qX  Q t ) , Chapter  Z,  Section  2.3,1. 

Assume,  for  the  problem  in  A2.2,  that 


I 1 1 

0 , otherwise 


Suppose  (u^J  ^ 0 is  larqe  enough  such  that 


(A2.3.1) 


i-.dlDb,  - i 

1 »0  i »0 


I 

( ‘>'-1— o-v 


b.  (A2.3.2) 


^ ( (273?)"k^,  “’I " ) 

i =0  0 


Similarly, 


P i, 
ik , 1 
0 


Then,  from  equation  (A2.2.14), 


Sj(ko)  - (a'^  + Dq  - a 


^ zl  ‘'ik  ‘'i 


(A2.3.3) 


(A2.3.4) 


(A2.3.5) 


(A2. 3.6) 


I 


= 2uQr  + 


)5o  ''0'°  k^O  ^’^0  ° 


+ 2ab,  x„)  S,  (A2.3.9) 

Kq  0 1 


Then, 


I 


k^O 

\ ,0  £ ^k  k ^k 

k^O  *^0  *^10  *^0  ^ 
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(A2.3.10) 


This  solution  is  valid  only  when  | Uq | >0  is  large  enough  such  that 
P((bj^  ^ ' 'Thus, 

I (bj^  -b^)  Uq+  CqI  >/3H,  (A2.3.11) 


must  be  satisfied. 

i)  Assume  (b,  -b.)  > 0.  Then  (A2.3.11)  is  satisfied  if 

*^0  ^ '"o 

(b.  -b.)  - /3t  > /3f  (A2.3.12) 

'^0  ^ “o 

or 

(b,  -b.)  > 2/3c  (A2.3.13) 

*^0  ^ *"0 


ii)  Assume  (b,  -b.)  < 0,  Then  (A2.3.11)  is  satisfied  if 

(b,  -b.)  + /3c  < -/3H  (A2.3.14) 

\ " “o 

or 

(b,  -b.)  < -2/3E  (A2.3.15) 

* 

Therefore,  must  satisfy 

I (b.  -b.)u*|  > 2/3c  (A2.3.16) 

' ko  1 0 ' 

for  (A2.3.10)  to  hold. 

* 

Notice  also  that  when  (A2.3.10)  is  the  optimal  solution,  is 
identical  to  the  deterministic  solution. 
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A2.4  Existence  ££  Steady-State  Solution  for  1-d  Example. 

From  Chapter  2,  Section  2.2,  the  coupled  Riccati  equations  for 


Sq  and  Sj^  are 


=o,t  ■ ’ * 


t+1 


+ p 


21 


t + 1 


(A2.4.1) 


®l,t  - 


"tPl2^^Vt+l  ^P22<"/^>^.t+l^ 


. p ( , - * P22'‘^>^.t.l' 

‘M  b * Pi2  ♦ ‘’22<‘^"'=2,t*l/ 


0,t+l 


(A2.4.2) 


Define 


k ' 


t S 


0,t 


(A2.4.3) 
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o^t 


t S 


0,t+l 


(A2.4.4) 


Dividing  both  sides  of  equations  (A2.4.1)  and  (A2.4.2)  by  S , 

VJ  f X 

manipulating  terms,  and  using  equations  (A2.4.3)  and  (A2.4.4)  yields: 

2 

, r[p,,ab  + P,,  (a/b)h^^,  1 

r = 


=0,t*l  =0,t*l  ♦ Piib"  * Pjj  r 


blPjj.b*  P2j(a/b)h|.^jl 


+ P 


21 


(*■ 


Pliab+  P2i(a/b)h^^^ 


^Opt+l’  ^ Pll*^"  ^ 


t+1 


(A2.4.5) 


t t S 


1 ^^^12^^^  ^ P22<^/^’ Vl^' 


0,t+l 


* >>12  r 


btp^^ab  t P22(a/b)h^^^l 


^ P22  r 


Pl2^^  ^ P22<^/^’\+1, 


‘’f<^/^0,ttl>  ^ ^12^’"'  ^ P22\+l/^" 


r)‘ 


t+1 


(A2.4.6) 
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Assume  , , S,  .-*■«>  as  t -*■  -oo  and  h.  -►  h,  F.  -►  F.  Then 

U/uXfU  t t 

/ btp^^ab  + P2j^(a/b)hl  \2 

' Pll^^  + P2i<l/b^h  / 


(p  ab  + p (a/b)h  \ 2 

a 2 ) h 

b[Pllb  + Pjih/b^l  / 


(A2.4.7) 


(b[pj^2ati  + P22(a/b)hl  \2 

a - - - I 

Pl2b  + P22(l/b'')h  / 


^ ^^22^  - 


P22^®/^^^  Y 

+ P„h/b^j/ 


^'^12^^  ^ P22 


■Pll 

^12 

.^21 

^22 

1-Pl  P2 


F = P 


b[pj^ab  + (1-p^)  (aA>)h)  \2 


/ b[pj^ab  + (1- 

^ \ p^b^  + (1-p^ 


) {aA>  )h 


+ (1-p,)  ( a - 


?^ab  + (1-Pj^)  (a/b)h  y 

2 2 I ^ 

atPjb  + (l-p^)h/b^]/ 


/ b[(l-p  )ab  + p (a/Tajh]  \2 

hr  - (i-p  ) . 2^ i— 

\ (1-P2)b^  + P2(l/b^)h]  / 


+ P2  a - 


(1-P2)ab  + P2(aA')h 
b[(l-P2)b^  + P2(l/b^)h] 


(A2.4.8) 


(A2.4.9) 


(A2.4.10) 


(A2.4.11) 


I 


A3.1  Proof  of  Theorem  1. 

Assume  for  Then  (B^-  B^)u^_^  * 0. 

which  implies  null  space  of  B “ B ^ , N(Bj^-  Bj^^  ). 

Now,  dimension  (N(B|^-  Bj^))  <m  because  the  B^^'s  are  distinct. 
Therefore, 


dimension  ( U N(B  -B  . ) ) < m 

k,£  ^ 

Therefore  the  set  U N(B  -B«  ) has  measure  zero  in  R 
k,ll  ^ 


(A3. 1.1) 


Q.E.D. 


••• 
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AJ,2  Optimdl  Solut  ion  for  D>’tt>rministic  Problem. 
For  the  system 


-k(t)  ‘ ^ -k  ^k-0 


Pit 


n ^ C R 


L+1 


(A3. 2.1) 
(A3. 2. 2) 
(A3. 2. 3) 


wl\ere  H ^ ■ probability  of  B . at  time  t. 
— 1 , t ~ 1 


Assume  that 


1)  X ^ is  observed  exactly 


Pk(t  1)  lk(t) 


3)  then  is  applied 


From  dynamic  programming,  the  optimal  cost-to-go  at  time  t is  given 
by 


-t 


V*(x^  ,k(t-l),t)  ■ min  ®k(t)  !-t^lt^  — t^-t 

+ V*(x^^j^  ,k(t),t+l)  I x^l 


(A3. 2. 4) 


Assume 


V (X ^ ,k(t-l) ,t) 


lllk.tlt 


(A3. 2. 5) 


Then 


T o 
X . S 


T I 

(AX^  (A3.2.h) 


i 


Li 
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and 


(A3. 2.6)  = min 


I 

“t-it'it'l 


X * fix  ^ + U Ru 


•t  -t--t 


Pik 


T T T T 

X^AS.  ^ ,AX^+U*B.S. 

— t l,t+l 1 — t — 1 — l,t+l  — i — t 


T T T T 

+ X^AS.^^,B.U.  + U.B.S...,Ax.  I,  TT. 

— t i,t+l— 1— t — t — 1— i,t+l t II  {A3. 2. 7) 

Differentiating  the  r.h.s.  of  (A3. 2. 7)  w.r.t.  and  setting  equal 

to  zero: 


0 = 


2 R u ^ + 


Pik 


2bTs.^,B.u^  + 2 bTs  . ....  A X ^ 

— 1 — i,t+l  — 1 — t — 1— i,t+l 1 


{A3. 2. 8) 


or 


k(t-l) ,t 


R + 


p..  B . S . B . 
ik  1 i,t+l  1 


-1 


{A3. 2. 9) 


is  the  optimal  , given  k{t-l) 


Since  no  noise  is  present  in  the  system,  k{t-l)  is  obtained  from 
X ^ and  X t-i  ' with  » as 


k{t-l)  = i iff  x^  = Ax^_j^  + 


(A3. 2. 10) 


Substituting  {A3. 2. 9)  into  {A3. 2. 7),  and  eliminating  x^  because  the 
equation  must  be  true  for  all  x^  and  the  matrix  equation  is  symmetric. 


r 


A3, 3 Proof  of  Lewna  1. 

Consider  the  optimization  of  the  cost-to-go  given  k(t-l)  at  time 
t with  final  time  T.  This  optimal  cost-to-go  is  simply 


^ fj.  t 


(A3. 3.1) 


where  T denotes  the  final  time.  For  the  process  with  final  time  T+1, 
the  optimal  cost-to-go  is 


V ^ (x  ^ ,k(t-l)  ,t) 


{A3. 3. 2) 


Since  this  optimal  sequence  is  not  necessarily  optimal  for  the  problem 
with  final  time  T,  it  must  not  incur  less  cost  over  {t,...,T}. 

V*^l  (x^  ,k(t-l),t) 


^ (x  ^ ,k(t-l)  ,t) 


I ij.  .j. 

+ E < u „ Ru  „ + X Q X 
T — T+1  x— 


(A3. 3. 3) 


Since  the  expectation  term  of  equation  (A3. 3. 3)  is  non-negative, 


V *^1  (X  ^ ,k{t-l)  ,t)  > V*  (X  ^ ,k(t-l)  ,t) 


(A3, 3. 4) 


Now,  note  that 

V*  (x^,k(t-l),t)  = x^S.^^^Xt 


(A3. 3. 5) 


and  that  equation  (3.3.6)  depends  only  on  the  number  of  iterations 


(T-t)  for  the  calculation  of  S . , and  therefore, 

vj.  (Xj.  (x^  ,k(t-l)  ,t) 


(A3. 3.6) 
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A5.1  Proof  of  Theorem  1,  Chapter  5. 

W 

Xzi 

I trlE^  (Q+^  RG)1  + tr[E^2l 


‘'t  = 


(A5.1.1) 


and  J < B.  Since  Q + RG  > 0 and  is  constant  for  all  t,  this  implies 
T a.  — 


lim  tr(^^]  = 0 


(A5.1.2) 


which  is  exactly  Definition  1. 

w 


From  equation  (5.4.6),  note  that 


(A5.1.3) 


where  F(‘)  is  linear  in  (E^  t'i=0' 
Since 


lim  tr  ) = 0 

t-^00 


(A5.1.4) 


for  any  choice  of  Eq  , ||  F ||  is  bounded  and  ||f  1|  < 1.  (Otherwise, 


}Io 

Then 


^ tr[E^  <2  + R G)]  + J trtE^2l  (A5.1.5) 


i tr[E^  ] ^ tr[2  + G^RG]  + ^ tr[E^]  ^r[2]  (A5.1.6) 


Il2+  gVII  +l|f'ir  Hioll  llsll  (A5.1.7) 


1 Xllf  II  ( II  ’ 


1- 


T+1 


WIqW  IIs  + RgI 


(A5.1.8) 

(A5.1.9) 


J 


A 5.2  Proof  of  Remark  on  Theorem  1,  Chapter  5. 


^T  " trlE^  (2  + G^RG^.)1+  tr[E^S)l 


^tr[E^2l  < 


Since  2^0 


tr[^^  1 is  bounded. 


Therefore 


tr[^^]-^0  as 


The  reverse  implication  is  shown  to  be  false  by  example* 


(A'j.2.1) 


(A5.2.2) 


(A5.2.3) 


(A5.2.4) 


Ex2unple  1:  Consider 


*t+l  ~ “t 
“t  "t 


- -ETT  ’'o 


(A5.2.5) 


{A5.2.6) 


(A5.2.7) 


(A5.2.8) 


• Example  1 is  provided  by  Dr.  D.  Caatanon  of  ESL. 


A5.3  Proof  of  Theorem  2^,  Chapter  5. 

Let  I = {o,  1,  2,  ....  l} 
and 

*,"(1)  = {{k(0),  k(l),  ...)|  k(i)G  l} 

00 

Define  the  function  (J  on  the  cylinder  sets  of  i,  (I) 
K = {{k(0),  k(l),  ...)|  k(i)  fixed  for  i^T} 
for  arbitrary  T by 


\(0)q  Pk(l)k(0)'  Pk{2)k(l) 
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(A5.3.1) 


(A5.3.2) 


(A5.3.3) 


(A5.3.4) 


k(T)k(T-l) 

where  tt  q is  the  initial  probability  distribution  over  I and  P = (P^j) 
is  the  stochastic  matrix  of  transition  probabilities  for  the  Markov 
chain.  By  a theorem  of  Andersen  and  Jessen  [Loeve,  p.91,421,  this 
function  defines  a measure,  y , on  the  o-algebra  of  generated  by 

00  oo 

the  cylinder  sets,  a (Z  (I)).  Since  JJ  (D)  = 1*  from  the  definition 

00 

of  p on  the  cylinder  sets  of  x,  (I), 

p;  a(Jl“(I))  [0,1]  (A5.3.5) 

is  a probability  measure,  and  since  p extends  uniquely  from  the  cylinder 

00 

sets,  it  is  the  probability  of  occurance  of  elements  of  a (Z  (I)). 


t,et 


J^(x)  ; R > [0,“] 


(AS. 3.6) 


^ — T 


where 


X -Ax  + u 

-t+1  --t  -t 


(AS. 3. 7) 


(AS. 3.8) 


X = (k(0),  k(l),  k(2),...) 


(A5.3.9) 


(A5.3.10) 


and  let 


J = lim  J 


(A5.3.11) 


Since  is  constant  on  the  cylinder  sets  with  fixed  sequences  of 
length  T+1,  is  measurable.  (There  are  a finite  number  of  such 
sets.)  By  Theorem  A of  [Halmos,  p.84,101,J  is  measurable  with  respect 


J(*)  (x)  : Z (I)-^  (O.ooj 


= {x  I J(x)  (x)  < oo  for  xe  r”  } 


(A5.3.12) 


(AS. 3. 13) 


= il  (I)  - X^ 


(A5.3.14) 


Then  Xj^  and  are  measurable  subsets  of  Z (I)  , and  therefore 


:(J]  < “ * y(X2)  = 0 


(A5.3.15) 


because  J(x)  is  a non-negative  function  on  R . 


E^[E[J1I  = tr(EgS^] 


(A5.3.16) 


from  equation  (5.7.14),  and  by  hypothesis,  r.h.s  (AS. 3. 16)  is  finite. 
Therefore,  any  trajectory  x is  an  element  of  with  probability  1, 


and  has  finite  cost. 


Therefore,  cost-stabilizes  (5.3.1)  with  probability  1.  Q.E.D. 


A5.4.  Proof  of  Theorem  3,  Chapter 


Notation ; Ir  <-he  proof,  the  sequences  (G  )“*  and  (G  )" 

— t t=0  —ns  t=0 

* t 

be  referred  to  by  G and  G respectively. 


Proof : 


I)  (=>)  Suppose  G is  cost-stabilizing.  Then  J(G 

— ns  ^ —ns 

* * * 

But  G minimizes  J.  Therefore,  J(G  )<  J(G  )=>J(G  )<  «». 

— — ns  — 

* 

Thus,  G is  cost-stabilizing. 


* * 

II)  (<=)  Suppose  G is  cost-stabilizing.  Then  J(G  )<  «> 

J{G*)  = lim  J (G^*) 

X-Hio 

Since  E U (G) ] = J„(G), 

X ns_  — T — 

T 


* * * 

J(G  ) = lim  E [J  (G  )J  = E tJ  (G  )] 


which  implies 


ns  — 


Since  G minimizes  J , then 
— ns  ns 

J (G  ) < J (G*)  < » 
ns  —ns  — ns  — 


and,  since  J = Jj,  for  all  T,  for  fixed  G, 

T 


J{G  ) < ®. 
— ns 


will 


) < ®. 


where 
(AS. 4.1) 


(AS. 4. 2) 


(AS. 4. 3) 


(AS. 4. 4) 


(AS.4.S) 


which  in^lies  that  G is  stabilizing 


O.E.D 
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A5.5  Proof  of  Lenina  £,  Chapter 

For  the  control  interval  starting  at  time  0 and  ending  at  time  T, 

* 

the  expected  cost  for  the  optimal  control  is 


T 


(A5.5.1) 


from  equation  (5.5.8),  where  the  subscript  T refers  to  the  endpoint 
of  the  control  interval.  Similarly,  for  the  same  process  ending  at 
T+1,  the  optimal  expected  cost  is 


(T+1)  RG*^(T+1) 


= E 


[£•:■=•  >v 

-T+1^  -T+1 1 -0  ' -oj 

X^  (2  + G*  (T+l)*^  R G*^(T+1)  X^ 

lo] 


(AS. 5. 3) 


+ E t Xt  - -T  -T  ^ ^T+1  S ^T+J  -0  '^^0  ’ 

(A5.5.4) 


The  first  expectation  of  equation  (A5.5.4)  is  the  cost  corresponding 

* 

to  the  interval  I0,Tl , and  must  be  greater  than  or  equal  to  the 

second  term  is  positive.  Therefore, 

j*  > j*  (A5.5.5) 

T+1  — T 

Since  is  bounded  by  hypothesis  for  all  T,  there  exists  a J such 

that 

lim  J*  = J*  (A5.5.6) 

T 

Q.E.D. 


I 


A5.6  Proof  of  Leirona  Chapter  5. 

By  direct  computation, 

and  since  the  expectation  is  positive. 


since  J_ (^)  is  bounded,  it  converges. 
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T+1 


2 X ] 

i.  - I 


(A5.6.1) 

(A5,6.2) 

Q.E.D. 


i I 
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A5.7  Proof  of  Theorem  4,  Chapter  5. 


A)  G 

— ns. 


-G  ^ because  G 
— opt  —ns 


converges  to  the  steady-state  value 


which  minimizes  the  inf inite-time  horizon  cost  J , and  therefore, 

ns 

ss 

by  the  argument  given  above,  also  minimizes  equation  (5.8.9). 


B)  Given  e > 0,  a T>  0 can  be  chosen  which  guarantees  [(  G ^ - G |(<  c, 

llZ-t  ~ 2LII  t > T. 

t * lO° 

■Then,  by  the  Principle  of  Optimality,  the  sequence j G^  't-T 

minimizes  the  infinite-horizon  cost-to-go  at  time  T.  Consider  the 

problem  min  J (G)  for  initial  condition  E . , tt  , which  has  a solution 
^ ss  - -1  - 

G^^  independent  of  . In  the  limit  as  e-K),  the  sequence  { G^  ^t-T(e) 

approaches  he  constant  sequence  of  gains  G . Suppose  j 6>0  3- V . , 

1 V ^ ' 


the  optimal  cost-to-go,  satisfies 

♦ * r 

< J - <5 

T(e)  — ss 


(A5.7.1) 


Then  the  sequence  of  constant  gains  G would  yield  a strictly  lower 


cost  J (G  ) 
ss  — 


* * 

J (G  ) < J 
ss  — ss 


(A5.7.2) 


since  -approaches  the  optimal  cost-to-go,  given  the  constant 

* 

sequence  of  gains  G , in  the  limit,  which  is  the  solution  to  the 

equivalent  problem  min  J (G)  for  initial  conditions  ^ 

^ ss  1 

Therefore 


G*  = G 
— —ns 


(A5.7.3) 


Q.E.D. 


n n n n 


AIM  FGRl'RAN 
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SUiROUTINE  AIM (NAA,  hA,  NQ,  NR,  NG,  NS,  M?A,  N, M,  ICCN,  A, B,  R,  0,  P, 

1 33T,  E,  S,  U,  V,W,  X,  Y,  FR,  PZ , QORM,  FAD,  FADIN V,  BSB, WCRK,  IPVT,  lEND, 

2 IPRT) 


*****PARAMErERS: 

INTECER  NAA,  MV,  ^B,  NQ,  NR,  N3,  NS,  FRA,  N,M,  ICCN,  IPVT  (N ) , lEND,  IPRT 
D0UBI£  HRBCISION  BSB  (NS,  FRA,  ICON) , X (NA,  N)  ,RAD  (NRA,  N)  ,RADINV  (NRA,  N) 
DOUBIE  PRECISION  E (KCCN)  ,SBT(NS,N)  ,A(NA,IRA)  ,B(N3,hRA,  iCCN) 

DOIBIE  PRECISION  Q(NQ,N)  ,R(NR,M)  ,P(NA,ICCN)  ,S(NS,FRA,  ICCN) 

DOIBIE  PRECISION  S3  (NS, FRA,  ICCN) , U (NA, N) , V(NA, N)  ,W(NA, N)  , Y (NA, N) 
DOUBIE  DECISION  PR  (N)  ,WORK (N)  ,re  (N)  ,QK)RM(NG,  FRA,  ICCN) 


C 

C 

C 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 


*****LOCAL  VARIABLES: 

DOUBLE  PRECISION  CCND 

INTECER  KIN , KDUT,  I,  K,  KKM 1,  KK,  J,  JEND,  L,  KP,  FM 1,  ICIM 1,  IM 1 
INTEOIR  ICOUNT 


*****SlBROUriNES  CALLED; 

MQF,MACD,MLLNEQ,  TRNATB, MM  DL,M9CALE,MATI0,  EIG\RL,  WEIGHT,  IRNATA 


*****PURFOSE: 

IHIS  DOUBLE  PRECISION  SDBROUrlNE  CCMRITES  THE  STEADY -STATE  OPTIMAL 
SOLUTION  AND  THE  CORRESPONDING  OPTIMAL  GAINS  FCR  THE  PROBLEM 
DESCRIBED  IN  TOE  PUBLICATION:  ' ON  TOE  REIATKDNSHIP  BETWEEN 
RELIABILITY  AFC  LINEAR  QUADRATIC  OPTIMAL  CONTROL ' 

BY  J.  DOUGLAS  BIRDWELL  AFC  M.  ATHANS. 

(EQUATIONS  (29)  AND  (30)). 


*****PARAMErER  DESCRIPTION: 

ON  INPUT: 

NAA  TOE  SBCa^D  DIMENSION  OF  THE  ARRAYS  S,SB,GNORM, 

BSB, B AS  DECLARED  IN  TOE  CALLING  PROGRAM 
DIMENSION  STATEMENT; 


FR,FB,NQ,NR,  THE  FIRST  DIMENSION  OF  THE  ARRA^ 

NG,NS,FRA  A (AND  P,  X, U,  V,W,  Y)  ,B( AND  BSB)  ,Q, R, (NORM, 

S (AFC  SB,33T)  ,RAD  (AND  RADINV)  RESPECTIVELY 
AS  DECLARED  IN  TOE  CALLING  PROGRAM  DIMENSION 
STATEMENT; 


N 


TOE  NUMBER  OF  STATES; 


M 


THE  NUMBER  OF  OBSERVATIOl'lS ; 


ICON 


TOE  NUMBER  OF  CONFIGURATIONS; 


A 


N BY  N SYSTEM  MATRIX; 


AIM  FORTtW^N 
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C 

C 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 


B 

R 

Q 

p 

E 


N BY  M BY  KCCN  SET  OF  INPUT  MATRICES; 

M BY  M CCNl’ROL  WEIGHTING  MATRIX; 

N BY  N STATE  WEIGHTING  MATRIX; 

KCCN  BY  KCCN  PROBABILITY  MATRIX; 

\«CTOR  OF  lENGTH  KCCN  CCNTAINING  THE  NORMALIZED 
EIGEN\CCTOR  OF  P CCRRES PONDING  TO  THE  EIGENVALUE 
ONE; 


O')  OUTHJT; 
l«,  PZ 


U,V,W,SBT, 

X,Y 


S 


SB,BSB 


GNORM 


RAD,RAn[NV 

WORK 


SCRATCH  VECTORS  OF  lENGTH  N; 

N BY  N SCRATCH  ARRAB  ; 

N BY  N BY  KCCN  SET  OF  SOLUTIONS; 

N BY  N BY  KCCN  SCRATCH  ARRATB  ; 

N BY  M BY  KCCN  ARRAY  WHICH  WILL  CCNTAIN  THE 
a IN  MATRICES  FCR  THE  NORMAL  LINEAR  QUACRATIC 
GVIBSIAN  PRCBIEM; 

N BY  N SCRATCH  ARRAB  ; 

SCRATCH  \CCTDR  OF  lENGTH  N; 


C 

C 

C 

C 

C 

C 

C 

C 

C 


IPVT  SCRATCH  \fiCTOR  OF  lENGTH  N; 

lEND  NUMBER  OF  ITERATIONS  USED  IN  SOLVING  BOTH  THE 

UNEAR  QUACRATIC  UBS  IAN  PROBIEM  AM)  THE 
IRCBIEM  [ESCRIBED  ABO\E ; 

IPRT  FIRST  ITERATION  AT  WHICH  THE  SOLUTIONS  Wlli  BE 

miNTED; 

CCN4CN/INOU/KIN,  iOUT 
ICONT  = 18 
DO  215  KK*1,ICCN 
DO  4 J>1,N 
DO  3 1*1,  N 

3 Y(I,J)*  0.0)0 

4 Y(J,J)«  1.0)0 
DO  210  K*1,IEND 

CALL  MCF(NA,{e,MV,N,M,Y,B(l,l,KK),U,WCRK) 


o n 
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CALL  MACD  MA,  NR,  hA,M,M,  U,  R,  U) 

DO  14  J=1,M 
DO  13  1*1, M 

13  V(I,J)=  0.eD0 

14  V(J,J)=  l.H)0 

CALL  MLINEQ(NA,NA,M,M,U,  V,CCND,  IPVr,WCRK) 

CALL  TRNA‘IB(NA,bB,N,M,B(l,l,KK),X) 

CALL  MMUL(NA,m,m,N,M,  N,  X,Y,  U) 

CALL  MMlJL(NA,hA,IA,N,M,N,U,A,  X) 

CALL  Mg'(NA,m,Myik,M,N,  V,  X,  RAD,WCRK) 

CALL  M9CAIE  t'lRA,  N,  N, -1.  eD0,RAD) 

CALL  MQF(NA,I«^,m,N,N,  Y,A,  U,WCRK) 

CALL  MACD  (NA,  ^A,  1^,  N,  N,  U,  Q,  U) 

CALL  MACD  (NA,  NRA,  MV,  N,  N,  U,  RAD,  Y) 

210  CCNriNUE 

KIWI  = KK  - 1 
WRITE  (KOUr,  44441) 

WRITE  (KOUT,  44442 ) KKM 1 
CALL  MATIO  (NA,  N,  N,  Y,  3) 

CALL  MMUL(NG,NA,MV,N,M,M,V,  X,Q*DBM(1, 1,KK)) 

CAIL  MSCAIE  (NG,M,N,-1.0O0,aiORM(l,l,KK)) 

CALL  MMLL(NB,NG,NA,N,N,M,B(1, 1,  KK)  ,GNORM(  1, 1, 1)  ,V) 
WRITE  (KOUT,  6000) 

CALL  MATIO(NG,M,N,GNORM(l,l,KK),3) 

CAIL  MACD(NA,l«V,MV,N,N,V,A,V) 

WRITE  (XOOr,  44443) 

CAIL  MATIO  (NA, N,  N,  V,  3) 

CALL  EIGVAL(NA,N,  V,V,IR,ra,WORK,IFVr) 

215  CONriNUE 
JEND*  1 

WRITE  fKOirr,  8000) 

CALL  MATIO  (NA,  ICCN,  ICa-l,  P,  3) 

DO  35  K*1,ICCN 
DO  30  J*1,N 
DO  40  1*1,  N 

S(I,J,K)*  0.®0 
40  CCNTINUE 
30  S(J,J,K)=  1.030 

35  CCNriNUE 

C START  ITERATION  TO  CAICLLATE  S(1),S (2),.  . .S(K),aDPr 
C 

C CAICLLATE  SB 
1 CCNTINUE 
DO  50  K*1,ICCN 

CAIL  MMLL  (NS,I©,  N5,M,  N,  N,S  (1, 1,  K)  ,B(1, 1,  K)  ,SB{1, 1,  K)) 
50  CONTINUE 

CAIL  WEIGHT(NS,N(VA,H:CN,N5,N,M,E,SB,ST) 

CAICUATE  RADICAL 


AIM  FORnVkN 


DO  55  K=l,iCCN 

CAIL  Mtf  (NS,hB,I©,N,M,S(l,  1,K)  ,B(1, 1,  K)  ,BS8(1, 1,  K)  ,WORK) 
55  CCNTINUE 

CALL  WEIGHT  (NB,  NAA,  ICCN,  NRA,M,M,  E,BSB,  RPiD) 

CALL  MADD  (NRA,  FAD,  R,  U) 

DO  54  J=1,M 
DO  53  1=1, M 

53  RADINVa,J)=  0.030 

54  RAD1NV(J,J)=  1.0)0 

CALL  MLINEQ(NA,NRA,M,M,  U,  RADINV,CCND,  IPVT,WCRK) 

C 

C CAICLLATE  NEW  SI,  1=1, 2, KZCN 

100  DO  1000  k=i,k:cn 

CALL  MMULa'JS,NRA,N^,M,N,M,®T,RADINV,U) 

CAU.  WEIGHT  <NS,NAA,  JCCN,NA,N,M,Pa,K),SB,  V) 

CALL  TRNA'IB(NA,NA,N,M,V,W) 

CALL  MMLL(NA,NA,M\,N,N,M,  U,W,X) 

CAU,  TRNA1B(NA,NA,N,M,U,W) 

CALL  MMLL(NA,m,hA,N,N,M,V,W,Y) 

CALL  MACD  (NA,  NA,  Wk,  N,  N,  X,  Y,  X) 

CAIi  M3CAIE  (NA,  N,  N,  -1 . 0)0,  X) 

CALL  TRNA'IA(NA,N,X) 

CALL  WEIGHT  (NA,  NAA,  ICCN,  hA,  N,  N,  P(1,K)  ,S,  V) 

CALL  MACD  CMA,  NA,  N^,  N,  N,  X,  V,  X) 

CALL  WEIGHT  (NB,NAA,  lCCN,m,M,M,  P(l,  K)  ,BSB,  Y) 

CALL  MACD  (NA,  NA,  Nik, M,M,Y,R,Y) 

CALL  MMUL  (NA,  NA,  m,M,  N,M,  U,  Y,  V) 

CALL  MMlL(NA,NA,m,N,N,M,V,W,Y) 

CALL  MACD  (NA,  AA,  hR,  N,  N,  X,  Y,  X) 

CALL  M(y  (NA,NA,NV,N,N,  X,A,U,WCRK) 

CALL  MACD  (NQ,  NA,  tG,  N,  N,  Q,  U,  S (1, 1,  K) ) 

1000  CONTINUE 

IF  (ICOUNT-IEfC)  11,12,12 

11  ICOUNT=  ICGUNT  + 1 
IF(ICOUNr.Lr.IPRT)  GO  TO  1 
ICIMI  = ICOUtfT  -1 

WRITE  (KOOT,  5000)  ICTMl 

DO  1005  k=i,h:cn 

IMl  = K-1 

WRITE  (KOUr,  4000)  KMl 

CALL  MATIO  (NS,  N,  N,  S (1, 1,  K) , 3) 

1005  CONl’lNUE 
O)  TO  1 

12  CONTINUE 
C 

C CCMfUTE  OPTIMAL  COST  FINCTION 

CAIL  WEiaiT(NA,N^A,iCCN,NA,N,N,E,S,  U) 

WRITE  r<<OUr,  7000 ) 

CAIL  MATIO  (NA,  N,  N,  U,  3) 


n n n n 
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QJ  K)  (23,  22),JEND 
CvJMHjrE  G opr 

23  CALL  MMLL(NA,NA,NA,N,M,N,W,A,  U) 

CALL  MSCAIE  WA,N,N,  -l.aD0,U) 
wRrrE  (Kour,  600a ) 

CALL  MATIO  (NA,M,  N,  U,  3) 

DO  217  KP=1,1CCN 

CAU.  MMLL(NA,^B,^A,N,N,M,B(1,  1,KP)  ,U,W) 

CALL  MACD(NA,NA,  m,N,N,A,W,W) 

CALL  EIG\AL(MA,N,W,W,»,PZ,WCRK,  IPVT) 

217  CCNTINUE 

CAICUAI’E  CCMIARISON  WIIH  OJORM 
ICOlJNr=  0 
DO  130  K=1,ICCN 
DO  120  J=1,N 
DO  110  1=1, N 

S(I,J,K)  = 0.0)0 
110  CONTINUE 
120  S(J,>3,K)  = 1.0D0 

130  CONTINUE 
JEND=  2 
400  CONTINUE 

DO  98  k=i,k:cn 

CALL  WEIGHT  (NS,  NA.A,  ICCN,  NV,  N,  N,  P(1 , K)  ,S,  U) 

CALL  MCF  (NA,  1^,  M^,  N,  N,  U,  A,  X,WORK) 

DO  % L=1,1CCN 

CAIL  MCF(NS,^e,^B,N,M,S(l,  1,  L)  ,B(1,  1,  L)  ,SB(1,  1,  L)  ,WORK) 
96  CONTINUE 

CAIL  WEIGHT  (NS,  mA,  ICON,  hA,M,M,  P(l,  K)  ,SB,y) 

CALL  MCf'(NA,Nft,NV,M,N,  Y,QWRM(1,  1,K),U,WCRK) 

CALL  MACD  (NA,  NA,  N\,  N,  N,  U,  X,  X) 

DO  95  l=i,k:cn 

CAIL  MMUL(NS,^©,N5,M,N,N,S(1,  1,  L)  ,B(1,  1,  L)  ,SB(1,  1,  L) ) 

95  CONTINUE 

CAIL  WEIGHT  (NS,  mA,  ICON,  hA,  N,M,  P (1 , K)  ,SB,  Y) 

CAIL  TRbATO(NA,NA,N,M,  Y,W) 

CAIL  'TRNATA(NA,N,  A) 

CALL  MMIL(NA,NA,  IA,M,N,N,A,  Y,  V) 

CAIL  MMtL(NA,NG,t«\,N,N,M,V,aJC«M(l,l,K),Y) 

CALL  MACD  (NA,  m,  N,  N,  Y,  X,  X) 

CALL  TRNAIB(NG,NA,M,N,Q*DRM(1,1,  K),V) 

CALL  MMUL(NA,^^\,^A,N,  N,M,V,W,U) 

CALL  TRt*VTA(NA,N,A) 

CALL  MMUL(NA,NA,  bA,N,N,N,  U,A,W) 

CAIL  MACD  (NA,  ^A,  MV,  N,  N,W,  X,  X) 

CALL  MACD  C^,  MV,  MV,  N,  N,  X,  Q,  X) 

CAIL  MCF(NR,NG,MV,M,N,R,CM0Rr4(l,  1,K)  ,U,WCRK) 
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AIM  FORTRAN  , 

CAU.  MACD  (NA,  NA,  N\,  N,  N,  X,  U,  X) 

CALL  SA\C  (NA,hB,N,N,  X,S(1,1,  K)) 

98  CONriNUE 

IF(ICOUN^-IE^D)  4010,4011,4011 

4010  ICOUNT=  ICOLNT  + 1 
GO  TO  400 

4011  WRirE  (KOOr,  9000) 

CALL  MCF(NA,^A,^A,N,N,  X,A,U,W(.ltK) 

DO  1006  l>i,k:cn 

IMl  = L-1 

^ WRirE  (KOUT,  4000 ) U11 

i:  CALL  MATIO  (NS,  N,  N,  S (1, 1,  L) , 3) 

1006  CCWriNUE 
00  TO  12 

4000  FORMAT  (/, 41  3,15,/) 

5000  FORMAT  (//,11H  ITERATIQ')  ,I3) 

6000  FORMAT  (//,10H  G OPPIMAL  ) 

7000  F(0RMAT(//,39H  OPTIMAL  COST  FINCTION  X^CX,  WICRE  C IS,/) 

8000  FORMAT!//,  21  P,/) 

9000  FORMAT  (//,38H  COST  CCMEARISON  WITH  NORMAL  SOLUTION  ) 

9500  FORMAT  (2D  25. 15) 

9600  FORMAT!/, 21  A ) 

9700  FORMAT!/,  21  Q ) 

9800  FORMAT!/,  21  R ) 

9900  FORMAT!/, 21  B,  15,/) 

44442  FC»MAT!/,21  S ,15,/)  , 

44443  FORMAT!/, 13H  A + B*GZERO)  I 

44441  FORMAT  !/,45H  SOLUTION  TO  STANDARD  OPTIMAL  CONTROL  PROBLEM)  ' ' 

2 STOP 
22  RCTLEUl 
END 


L 


no  nn 


SWrrCH  FCRTRAN 
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C 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 


SLBROUriNE  SWITCH  CSIA,^«,^C,NG,N^R,  me,  N,  IR,  mA,  1CCN,M,  A,B,  P,C,G, 
1X0,  E,  LTEMP,  EM,  WCRK,  Y,  U,  V,W,  W,  IPVT,  ARRAY,  DT,  NPOIWI,  NGRIDH,M^M'I ) 

*****PARAMETERS: 

INTEGER  NA,ie,N:,mR,mC,N,  IR,mA,lCCN,M,NFOINT,NG 
INTEOiR  MCa^  (NPOINI)  ,IPVr  (N) 

DOUBIE  PRECISION  A(NA,N)  ,B(NB,NAA,  ICCN)  ,C(NC,N)  ,X0  (N) 

DOUBI£  mBCBlON  G (NG,NAA,  ICCN)  ,Y(N) , WCRK  (N)  ,EM(NA,N) 

DOUBLE  PRECISION  U (M)  , W(NA,  ICCN)  ,W(NA, N)  , V(NA, N) 

DOUBIE  PRECISION  ARRAYINAR,  mC)  ,P(NA,  ICCN)  ,E  (KCCN)  ,ETEMP(KC(N) 

*****L0CAL  VARIABIES: 

INTECER  IN(27),NSW(l),rr(10,l) 

DOUBLE  PRECISION  WT(IO) , SIM,  WOPI,  WIN,  WAX,  ISF(10) , ZERO,  »<AX,T,  DT 
DOUBIE  PRECISION  DD 
DIMENSION  R(30) 


***t**SlBROUriNES  CAILED: 
MMUL,M.SCAI£,MEXP,  SA\«,  FIG,  TOFLT 


*****FINCTIU'JS: 

GGUB,UCAIC 


*****PURR)SE; 

•nils  DOIBIE  PRECISION  SLBROUTINE  PERFORMS  TOE  COM PWATIONS 
AMD  PRINTS  THE  CftTA  FCR  SIMUATION  OF  TOE  SWITCHING  GAIN 
IROBIEM  RELATING  ’lO  TOE  PIBLICATION:  'ON  THE  REIATIONSHIP 
BBIWEEN  RELIABILITY  AM)  LINEAR  QUACRATIC  OPTIMAL  CONTROL* 

BY  J.  DOUGLAS  BIRCWEIE  AM)  M.  ATOAI6. 

*****PARAMETER  DESCRIPTION; 

m,IB,  NC,NG,  TOE  FIRST  DIMENSION  OF  TOE  ARRAW  A (AND  EM, 

NAR  W,W,V),B,C,G,AM)  ARRAY  RESPECTIVELY  AS 

CECIARED  IN  THE  CALLING  PROGRAM  DIMENSION 
STATEMENT; 

me  COLUMN  DIMENSION  OF  TOE  ARRAY  CONTAINING  ARRAY 

AS  DECLARED  IN  THE  CALLING  PROGRAM  DIMENSION 
STATEMENT; 

N NUMBER  OF  STATES; 

IR  NUMBER  OF  OOTIVTS; 

NAA  TOE  SECOND  DIMENSION  OF  TOE  ARRAW  B AM)  G AS 

CECIARED  IN  TOE  CALLING  PROGRAM  DIMENSION 
STATEMENT ; 


( 


SWriCH  FOKl’RAN 


THIRD  DIMENSION  OF  THE  ARRAC  B AND  G AS 
CECIARED  IN  THE  CALLING  PROGRAM  DIMENSION 
SrATEMENT; 

NUMBER  OF  CCNTROLS ; 

N BY  N 3BTEM  MATRIX; 

N BY  A BY  KCCN  SET  OF  OUTRTT  MATRICES; 

IR  BY  N OUT  PUT  MATRIX; 

M BY  N BY  KCCN  SET  OF  FEEDBACK  MATRICES; 

INITIAL  CCNDITION  VECTOR  OF  IZNGTH  N ; 

VECTOR  OF  lENOm  NBOINT  CONTAINING  THE  EWkCT 
CCNFIGURATION  INDICES; 

SCRATCH  VICTOR  OF  LENOm  NCCN ; 


BIEMP 


SCRATCH  \CCTDR  OF  LENGTH  KCCN ; 


SCRATCH  \fiCTOR  OF  lENGTH  N; 
VECTOR  OF  LENGTH  N; 

VECIOR  OF  lENGTH  M; 


V,W,VW,EM 


N BY  N SCRATCH  ARRAS  ; 


SCRATCH  \«CTOR  OF  lENGTH  N ; 


ARRAY 


NAR  BY  NAC  WGRKING  ARRAY; 

NAR  MLBT  BE  (3IEATER  THAN  OR  EGUAL  TO  NSTERj  + 1 
NAC  MLBT  BE  ®EATER  TRAN  OR  EQUAL  TO  IR  + M ; 


STEP  SIZE; 


NHOINT 


NUMBER  OF  STEPS  + 1; 


NGRIDH 


NUMBER  OF  MAJOR  ORDINATE  DIVISIONS  LBED 
IN  PLOTTING 

NGRIDH  MLBT  BE  I£SS  TRAN  C«  EQUAL  TO  12; 


*****NOTES: 

BOTH  THE  OGTIVT 


AMO  THE  CCNTTROL  U (T)  » -G(I)*XfT)  ARE  CCMR/TED. 


t 
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aw ITCH  FORTRAN 


OUB  IS  A RAMXM  NIMBER  GENERAIUR 

UCAIC  IS  A LEER -SUPPLIED,  APPLICATION  SRCfflC  FUNCTION  TO 
CALCULATE  THE  COfTROL  U. 


*****HISIORY: 

WRirreN  BY  J.A.K.  CARRIG  (El£C.  SC.  lAB.,  M.I.T.,FM.  35-3!37, 
CAMBRIDGE,  MA  02139,  HI.:  (617)  - 253-2165),  OANUfVRY  1978. 
MOST  RECENT  VERSION;  MARCH  22,  1978. 


COWCN/INOU/KIN,  KDUT 
LCGN  * 1 

DATA  ^F/10*1.(D0/,IBIANK/4H  / 

DATA  ■IWGPI/3.  1459/ 

DATA  M3:,MA)4:S,  IXY,  lEGY, ZERO.MM,  NDS,  IZERO/1, 0,  0,  1, 1.  0D0,  1,  0,  0/ 
UATA  IN(1),IN(2),IN(3),IN(4)/4H1  ,412  ,413  ,414  / 

DATA  IN  (5)  ,IN(6)  ,1N(7)  ,1N(8)/4H5  ,416  ,417  ,418  / 


DATA 

IN  (9), IN  (10), 

IN(11),1N(12)/4H9 

,4110 

,4111 

,4112 

/ 

DATA 

IN  (13) ,1N  (14) 

,IN(15)  ,IN(16)/4H13 

,4114 

,4115 

, 4116 

/ 

DATA 

IN  (17)  ,IN(18) 

, IN  (19) , IN  (20)/4Hl7 

,4118 

,4119 

, 4120 

/ 

DATA 

IN  (21 ) , IN  (22 ) 

,IN(23),IN(24)/4H21 

,4122 

, 4123 

,4124 

/ 

DATA 

IN  (25),  IN  (26) 

,IN(27)/4rt25  ,41 

Y,41 

U/ 

D,\TA 

IT  (3,1),  IT  (4, 

1)  ,IT  (5,  l)/4riVERS,  41US  T,  4iIME  / 

DATA 

n’(6,  1),1T  (7, 

1),IT(9,  1)/4H  ,41 

,41 

/ 

DATA 

n’  (9,  1),1T(10 

,1)/4H  ,41  / 

IX*35 

DU  61  IZ*l,NPOINT 
61  MCCN  |IZ)=  MCON  (IZ)  + 1 
■IWOPI  * 2.a)0*IWOPI 
NSTER;*  --  NIOINT  -1 
T»  0.  J)0 

3001  FORMAT  (24H  EJftCT  CCNFIGURATION  = ,13) 

CALL  MMUL|NC,N,N,MM,  1R,N,C,X0,  Y) 

CALL  MMUL(NA,N,M,MM,M,N,G(1,  1,MCCN(1))  ,X0,U) 
WRITE  (KOUT,  1500) 

WRITE  (KUUr,  1200 ) 

WRITE  (KOUT,  1300) 

WRITE  (XOUT,  1000)  T 
1031  FORMAT  (/,12H  GAIN  MATRIX) 

WRTTE  (KOUT,  1100)  (Y  (I ) ,1=1,  IR) 

WHTTE  (KOUT,  1102)  (U(I),I»1,M) 

C WRITE  (KOUT,  1001) 

30  ARRAY (1,J)-  Y (J) 

DO  40  J»1,M 
40  ARRAY(1,1R-U)-  U (J) 

50  DO  100  K-1,NS'TERJ 


fV?/2l  * ^(li*ri^^^ 


S3i  pl^  un^ 

in*  ^or  ?'-^/?i  p p^ 

^00iT  - * ' 

P^^  (KOOp^^  «+i  J * 

r?^T(/^  CAicrw^Otij 

“0  %J!?5  «?;/,'*•'«.  “^riav  , 


Sf^'S(r  . ®''«n,j  ,, 

(^ui  , 

^ S0  t V '^*^>M) 
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SWITCH  FCRPRAN 

JMAX  = DF  LOAT  (NSTEPS ) 

IW=  Kour 
NSW(1)=  25 
IT  (1,1)=  IN  (26) 

DO  110  J=1,IR 

IF(J.I£.  25)  IT  (2,1)=  IN(J) 

IF (J. or.  25)  IT  (2,1)=  IBIANK 

110  CALL  THUiT  (IW,IECT,NFOIMr, ZERO,  »1AX,N®IDH,VMIN,  WAX,  «F,rr, 
1 ARRAY(1,J),NAR,NLG,MSC,MAXES,  IXy,NSW) 

IT  (1,1)=  IN  (27) 

NSlM(l)  = 21 
DO  120  J=1,M 

IF(J.t£.  25)  IT  (2,1)=  IN(J) 

IF(J.Gr.  25)  IT(2,1)=  IBIANK 

120  CALL  THFLT  (IW,IEGY,NroiWr,ZER0,X4AX,NGRIDH,  WIN,  WAX,WF,  rr, 
1 ARRAY  (l,J+IR),NAR,NLG,MSC,MA»:S,I»f,  NSW) 

1100  FORMAT  (4H  Y = , 5 (2X,  1PD19.  8) ) 

1000  FCRMAT(5HT=  ,F5. 2) 

1102  FCRMAT(4H  U = , 5 (2X,  1PD19.  8) ) 

1200  FORMAT  (llH  OOTRJT  Y) 

1300  FORMAT  (12H  CCNTROL  U) 

1400  FORMAT  (/,28H  3IMUATION  OP  LINEAR  S'BTEM,/) 

1500  FORMAT  (/,31H  SIMUATION  OP  LINEAR  REGULATOR,/) 

RETURN 

END 


on  on 
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READY  FORl'RAN 

SlBROirriNE  READY  (NAA,  N\,  t©,  NQ,  NR,  NG,  NS,  hRA,  N,M,  ICCN,  A,B,  R,  0,  P, 
1WR,W  1,  S,  33,  U,  V,W,  X,  Y,  OJORM,  HAD,  RADINV,  BSB,WCRK,  IPVr,  lEND,  NSTEPJ ) 

*****PARAME1'ERS: 

JiTECER  NAA,N\,N3,NQ,I«,hG,NS,hRA,N,M,lC0N,  IPVrW) 

DOUBLE  DECISION  A (NA, N) , X WA, N)  ,Q(NQ,  N)  ,R (NR,M) 

DOUBLE  PRtJCIBIUM  S (NS,  NAA,  ICCN) , P (NA,  ICON)  ,SB|NS, NAA,  ICCN) 

DOUBLE  PRECISION  CMORM(NG,  NAA,  ICCN)  ,BSB(N3, NAA,  ICON)  ,WR (N)  ,WI  (N ) 
DOUBLE  mBCISION  B (N3, NAA,  ICCN) , RAD (NRA,N)  , RADINV (NRA,N) 

DOUBIE  H3ECISION  U (NA,N) , V(NA, N)  ,W(NA, N)  , Y (NA,  N)  ,W0RK  (N ) 


C 

C 

C 

C 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 


*****L0CAL  VARIABIES: 

EUUBIE  PRECISION  CCND 

INTEOiR  KIN,  lOUT,  KL,  Wll,  J,  1,  K,  JEND,  hCND,  L,  IN  1 


*****SU3R0UriNES  CALLED: 

MCF , MADD,  M UNEO,  IRNA'IB , MM  UL,  MSCA  IE , EIGVAL,  SANE , W EIGHT 


******PURIOSE: 

■IHIS  DOUBIE  PRECISION  SDBHOUriNE  SOLNES  TOE  SWITCHING -GAIN  PRCBIEM 
RELATING  TO  THE  RBLICATia'l:  'ON  THE  RELATIONSHIP  BETWEEN 
RELIABLILITY  AND  LINEAR  QaALRATIC  OPTIMAL  CONTROL ' 

BY  J.  DOUGLAS  BIRCWELL  AND  M.  ATHA1«. 


“♦“PARAMETER  DESCRIPTION; 
ON  INPOT: 
lAA 


TOE  SECOND  DIMENSION  OF  TOE  ARRAS  S,SB,GNORM, 
BS8,B  AS  DECLARED  IN  TOE  CAIEING  PROGRAM 
DIMENSION  STATEMENT; 


NA,N3,NQ,NR,  'IHE  FIRST  DIMENSION  OF  TOE  ARRAS 
NG,NS,MV3  A (AND  P,  X,  U,  V,W,  Y)  ,B( AND  BSB)  ,0, R, Q^RM, 

S (AND  SB)  ,RAD  (AND  RADINV)  RESFECTIVELY 
AS  DECLARED  IN  THE  CAUSING  PROGRAM  DIMENSION 
STATEMENT; 


N 

M 

iCON 

A 

B 

R 


TOE  NUMBER  OF  STATES; 

TOE  NUMBER  OF  OBSERVATIONS ; 

THE  NUMBER  OF  CONFIGURATIONS ; 

N BY  N 3STEM  MATRIX; 

N BY  M BY  KCQ'J  SET  OF  INPLTT  MATRICES; 
M BY  M CCNTROL  WEIGHTING  MATRIX; 


non 
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REACV  FORTRAN 


C 

C 

C 

C 

c 

c 


c 


C 

c 

c 

c 

c 


c 

c 

c 

c 


c 

c 

c 

c 

c 


c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 


Q 

p 


N BY  N STATE  WEICSITING  MATRIX; 
KCON  BY  KCCN  PROBABILITY  MATRIX; 


ON  oormr: 

WR,WI 


s 

SB,B,BSQ 

U,V,W,X,Y 

Q«RM 


SCRATCH  VECTORS  OF  lENGIH  N; 

N BY  N BY  RCCN  SET  OF  SOLLfTIONS ; 
N BY  N BY  KCCN  SCRATCH  ARRAS ; 

N BY  N SCRATCH  ARRAS  ; 


N BY  M BY  KCCN  ARRAY  USED  TO  STOPJ:  THE 
Q>iIN  MATRICES  FCR  THE  NORMAL  LINEAR  QUACRATIC 
aiBSIAN  PROBIEM.  ON  RETURN,  GNORM  CCNTAINS  THE 
OklNS  ASSOCIATED  WITH  THE  SWITCHING  GAIN  PROBIEM; 


RAD,RAniNV 

WCRK 

IPVT 

TEND 


N BY  N SCRATCH  ARRAS  ; 
SCRATCH  \«CTOR  OF  lENGTH  N ; 
SCRATCH  \BCTOR  OF  LENGTH  N; 


NUMBER  OF  IIERATIONS  USED  IN  SOLVING  THE  NORMAL 
LINEAR  QUAIRATIC  aLBSIAN  PROBLEM; 


^eTE^B 


NUMBER  OF  TIME  STEPS  LBED  IN  COMRITING  S 


*****NOTES: 

THE  SOLUTIONS  TO  THE  NORMAL  LINEAR  QUACRATIC  PROBLEM, 

■IHE  EIGENVALUES  OF  IHE  MATRICES  (A  + B(I)*GNORM(ERO) ) 

AS  WELL  AS  THE  EIGENVALUES  OF  THE  MATRICES  (A  + B(I)*G(I)) 
ARE  PRINTED. 


♦♦♦♦♦HISTCRY; 

WRITTEN  BY  J. A.  K.  CARRIG  (EIEC.  SS.  lAB.,  M.I.T.,  IM.  35-307, 
CAMBRIDGE,  MA  02139,  PH. : (617)  - 253-2165) , JANUARY  1978. 
MOST  RECENT  VERSION;  MARCH  22,  1978. 


CChMCNANOU/KlN,  KOUT 
WRITE  (KOUT,  9600) 

CALL  MATIO  |NA,  N,  N,  A,  3) 
WRITE  fKOUT,  9700 ) 


READY  FORTRAN 


CAIL  MATIO  P4A,  N,  N,  Q,  3) 

WRITE  (KOOT,  9800) 

CALL  MATIO  WR,  N,  N,  R,  3) 

DO  222  KL»l,iCCM 
IMl  » KL-1 

WRITE  (KOOT,  9900)  KMl 

CALL  MATIO(^^a,N,M,B(l,l,N.),3) 

DO  4 J=1,N 
DO  3 1=1,  N 

3 Y(I,J)*  0.0)0 

4 Y(J,J)=  l.£D0 
DO  210  K=1,1£MD 

CAIL  M(^’(NA,^©,^A,N,M,y,B(l,  1,  KL)  ,U,WCRK) 

CALL  MADD  (NA,  hR,  U,  R,  U) 

DO  14  J=1,M 
DO  13  1=1, M 

13  V(I,J)=  0.  0D0 

14  V(J,J)=  1.0)0 

CALL  MLINEQ{^IA,^ft,M,M,U,V,COND,  IPVr,WCRK) 

CAIL  TRNATB(NB,MV,N,M,B(1, 1,KL)  ,X) 

CAIL  MMUL  (NA,  ^A,  m,  N,  M,  N,  X,  Y,  U) 

CALL  MMUL(NA,hA,NV,N,M,N,U,A,X) 

CALL  M(J‘(NA,NA,M^,M,N,V,X,W,WQRK) 

CALL  M9CAIE  «A,N,N, -1.  a)0,W) 

CALL  MGF(NA,hA,MV,N,N,  Y,A,  U,WCRK) 

CALL  MACD  (NA,  tA,  m,  N,  N,  U,  Q,  U) 

CAIL  MACD  (NA,  hA,  N,  N,  U,W,  Y) 

210  CCNTINUE 

WRIFE  (KOOT,  44441 ) 

WRITE  (KOUT,  44442) 

CALL  MATIO  (NA,  N,  N,  Y,  3) 

CAIL  MMCL(NA,MV,NS,N,M,M,V,  X,(H0RM(1, 1,KL)) 

CALL  MSCAIE  (NA,M, N, -1.  0)0, aWRM(l,  1,KL)) 

WRTFE  (XOUT,  6000 ) 

CALL  MATIO  (NG,M,N,agC«M(l,  1,KL)  ,3) 

CALL  MMlL(N3,NG,tA,N,N,M,B(l,  1,IL)  ,GN0RM(1, 1, 1)  ,V) 
CALL  MADD  m,  NA,  WV,  N,  N,  V,  A,  V) 

WRITE  r«K)Ur,  7008) 

CALL  EIGVAL(NA,N,  V,V,WR,WI,WCRK,  IPVT) 

222  CONTINUE 
JEND*  1 
26  CCNTINUE 

WRITE  (XOUT,  8000) 

CAIL  MATIO  WA,  ICON,  ICCN,  P,  3) 

DO  5 k-i,k:cn 

CAIL  SAVE  (NQ,NS,N,N,Q,S(1, 1,K)) 

5 CCNTINUE 

DO  91  NEND=  1,NSTEP5 
WRITE  (KOUT,  4500)  NEND 


BfcADY  FORTRAN 


20»)  COMTINUE 

DO  90  l=i,k:cn 


DO  80  K=1,1CCN 

CALL  M(F(NS,l>©,t©,N,M,S{l,  1,K),B(1, 1,K),BSB(1, 1,  K)  ,WCRK) 
CALL  MMUL(NS,^©,^6,M,N,N,S(1,  1,  K)  ,B(1,  1,  K)  ,SB(  1, 1,  K) ) 

80  CONTINUE 

CAIL  WEIGHT  (NS,  ^ftA,  ICa-J,  ^R,  N,M,  P(l,  L)  ,S8,  V) 

CALL  WEIGHT  (NS,  ICCN,  hA,M,M,  P (1,  L)  ,BSB,  RAD) 

CALL  MACD  (NR,  t«A,  ^A,M,M,  R,  RAD,  U) 

DO  98  J«1,M 
DO  97  1*1,  M 

97  RADINV(I,J)  » 0.CD0 

98  RADINV(J,J)*  1.000 

CALL  MLINEO(NA,MiA,M,M,U,RADINV,CCND,  IPVT,WCRK) 

DO  70  K=1,1CCN 
DO  60  J=1,N 
DO  60  1=1,  M 


60  BSB(I,J,K)  = SB(J,I,  K) 

70  CONTINUE 

CALL  WEICHT  (NS,  MVA,  JCON,  m,M,  N,  P(1,L)  ,BS8,U) 

CALL  MMULCJRA,NA,M>k,N,M,M,HADINV,U,W) 

CALL  MMUL(NA,NA,N>k,N,N,M,V,W,  Y) 

CALL  MMUL(NA,M^,  NG,N,M,  N,W,A,Q«:«M(1, 1,L)) 

CALL  M3CAIE  (NG,M,N,  -1.  eD0,(MORM(l,  1,L)) 

LMl  * L-1 

WRITE  fKDUr,  2005)  UM 

CALL  MATIO  (NG,M,N,GNORM(l,  1,L)  ,3) 

IF(NEND.^E.NSTEH^)  GO  TO  73 

CALL  MMLL(NB,NG,I^,N,N,M,B(1,  1,L),QWRM(1, 1,L)  ,W) 
CALL  MADO(NA,NA,IA,N,N,A,W,W) 

WRITE  (KOUT,  7009 ) LM 1,  IM 1 

CALL  EIGVAL(NA,N,W,W,WR,WI,WCRK,  IPVT) 

7 3 CALL  MSCAIE  (NA,  N,  N,  -1 . 050,  Y) 

CALL  WEIGWT  WA,  NAA,  fCCN,  NA,  N,  N,  P{1,  L)  ,S,W) 

CALL  MACD(NA,M\,NV,N,N,W,Y,Y) 

CALL  MCF(NA,NA,M^,N,N,  Y,A,W,WCRK) 

CALL  MACD  (NA,NA,  re,N,N,W,Q,S(l,  1,L)) 

WRITE  (KOUT,  4000)  ml 

CALL  MATIO  (N3,N,N,S(1, 1,L)  ,3) 

90  CONTINUE 

91  CONTINUE 

2000  FORMAT  {3D25.  15) 

4005  FORMAT  (3H  S) 

2005  FOR4AT(4H  G,  13) 

4000  FORMAT  (4H  S, 13) 

4500  FORMATdlH  TIME*  T 2 -,  13) 

5000  FORMATdlH  ITERATION  ,13) 

6000  FORMAT  (10H  G OPTIMAL  ) 

7000  FORMAT  (40H  OPTIMAL  COST  FLNCTION  X C X,  WfCRE  C IS) 
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7003  FOR4AT(21H  A + 3(1  )*G3TAR  (ZERO) ) 

7009  FORMAT  «7H  A +3,13,31  * G,  13) 

8000  FORMAT  (3H  P) 

9500  FORMAT  (3D 25. 15) 

9700  FORMAT  (3H  Q) 

9600  FORMAT  (3H  A) 

9300  FORMAT  (3H  R) 

9900  FORMAT  (3H  B,  13) 

44441  FORMAT  (/,45H  SOLUTION  TO  STAM)ARD  OPTIMAL  CONTROL  PROBIEM) 
2 STOP 

44442  FORMAT  (3d  S ) 

RETLRN 

e:nd 


on  on 
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S IBRUUr  INE  W EIGHT  (NA , NAA , JC CN , NX,  N,  M , E,  A , X ) 
INl'EOiR  r^,tAA,K:CN,NX,N,M 

IIXJOIE  PRECISION  E (KCCN)  ,A(NA,N^A,^CGN)  ,X(NX,M) 


i 

i 

i 


C 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

r> 

C 

C 

c 

c 

c 

r> 

c 


*****I>0CAL  WVRL^BIiiS: 
IMl’ECER  I,J,K 
D0UB1£  PRECISION  SIM 

*****SUBROUriNES  CAUED; 
NONE 


*****PURP0SE: 

'IHIS  SLBRDUriNE  CCMRJTES  THE  WEIGHTED  SIM 
SLmATION  E(I)*A(I,0,  K);  1=1, N;  J=1,M;  K=1,1CCN. 

*****PARAMEI’ER  INSCRIPTION: 

N^  THE  FIRST  DIMENSION  OF  IHE  ARRAY  A AS  DEXTIARED  IN 

■IHE  GAINING  PROGRAM  DIMENSION  STATEMOH’; 

^AA  THE  SECCND  DIMENSION  OF  THE  ARRAY  AS  DECIARED  IN 

THE  GAINING  PROGRAM  DIMENSION  STATEMENT; 

ICCN  THE  THIRD  DIMENSION  OF  THE  ARRAY  A AS  DECLARED  IN 

THE  GAINING  PROGRAM  DIMENSION  STATEMENT; 

NX  THE  FIRST  DEMONS  ION  OF  THE  ARRAY  X AS  DEGIARED  IN 

GAINING  PROGRAM  DIMENSION  STATEMENT; 

N THE  ROW  SIZE  OF  A; 


C M THE  CCLLMN  SIZE  OF  A; 

C 

C E \«C  IDR  OF  LENGIH  KCCN  ; 

C 

CAN  BY;  M ARRAY 
C 

C *****HIS'IURY; 

C WRITTEN  BY  J.  A.  K.  CARRIG  (BlfC.  SYS.  lAB. , M.I.T.,  RM.  35-307, 
C CAMBRIDGE,  MA  02139,  HI.:  (617)  - 253-2165),  JANUARY  1978. 

C MOST  RECENT  \«RSION  MARCH  22,  1978. 

C 

C :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 

C 

DO  10  J*1,M 
DO  10  1=1, N 
X(I,J)  «0.eD0 


t 


( 
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DO  10  K*1,ICCN 

10  X(I,J)  * X(I,J)  + E(K)*A(I,J,K) 

RETURN 
END 


I 

! 


on  on 


I i 


1 
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C 

C 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 


FORl’RAN 

SUBWJUriNE  FlG(KCa4,E,Bl’EMP,WCRK,lCCN) 


*****FARAMErERS; 

DOl«l£  PRfcL'ISlJN  WCRK(KCCN)  ,E(KCCN),ErrEMF(KCGN) 


*****LOCAL  VARIABLES; 

INl’EQiR  MM,  LrEMF,  IF  LAG,  KK,  IF,  lU 
DOUBLE  PRECISION  SU4 


*****SlBWJUriNES  CAllED: 
NONE 


*****FURPOSE: 

IRIS  DOUBIE  IRBCISIJN  SUBROUriNE  IS  lEED  IN  HWOIRESIS  TESTING. 
AT  EACH  TIME  T,  ONE  OP  ICCN  HVPOrHESES  IS  CHOSEN. 


RHO(X(r)  - A*x(r-i)  - B(i)*ufr-i))*pi  fr-iA-i) 

I 


PI  (r-iA) 
1 


SCW(RHQ(Xfr)  -B(J)*Ufr-l))*PI  fr-l/T-1) 

j 


HYPOIRESIS  H(I)  IS  ASSIMED  TO  BE  CCRRBCT  IF 


PI  fT/T-l ) > PI  (F-l  A)  FOR  AIL  J NCn*  EQUAL  I 
I J 


TIES  ARE  RESOLVED  ARB IFRARILY. 

RHO{X)  DENOI’ES  IRE  PROBABILITY  DISTRIBUTION  OF  X. 


♦•••‘PARAMETER  CESCRIPTION: 
CN  INWT: 


ICON 

E 

WORK 


THE  NUMBER  OF  HYPCrniESES ; 

\EC1UR  OF  LENGTH  KCON  CONTAINING  PI  fT-lA-1); 


VECTOR  OF  LENGTH  NCCN  CONTAINING 
RHO(X(T)  - A‘X(T-1)  - B(I)‘UfT-l)); 


CN  OUTFGT: 
ETEMP 


ICON 

•••HISIORY; 


VECTOR  OF  LENGTH  KCCN  TO  S'TORE  PI  fT/1-1); 
INDICATES  WHICH  HYPOHIESIS  HAS  BEEN  CHOSEN; 
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C WRin’EN  BY  J.A.K.  CARRIG  (ELEC.  SIS.  lAB. , M.I.T.,  R4.  35-307, 

C CAMBRIDGE,  MA  02139,  fH. : (617)  - 253-2165),  JANUARY  1978. 

C MOST  RBCEWr  VCRS  ION  MARCH  22,  1978. 

C 

C 

C(>mcnanouain»  HDur 

MM  » 1 
LTEMP  » LCCN 
SIM  0.  eD0 
DO  10  IP  * l,iCCN 
10  SIM  = SIM  + WORK(IP)*E  (IP) 

DO  20  IP-1,  ICCN 

20  ETEMP(IP)  » WORK(IP)*E  (IP)/S(M 
DO  60  KK  « 1,ICCN 
IFIAG  - 0 
DO  89  IU-  i,k:cn 

IF(KK.BQ.  lU)  GO  TO  79 
IF(ETEMP(KK).Gr.E(IU))  IFIAG  - IFIAG  + 1 
79  CCWriNUE 
89  CGNTINUE 

IFIAG  * IFIAG  1 
IF  (IFIAG.  EQ.  ICON)  LCGN-  KK 
60  CONTINUE 

IF(LCCN.EQ.  0)  LCON  « LTOiP 

REFIRN 

END 


men 
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C lA'l’EST  \CRS10N  3/9/77 

D0UB1£  {VISION  CCND,BEE,WR(10),W1(10) 

DOUBIE  reeCISlON  A(10,  3),X(10,  3) 

INTEGER  MDQM»(2),HRMI6C(2),VriME(2),RriME(2) 

[X)U31£  PRECISION  GNORM(10,  3,4) 

OOUBIE  PRECISION  3SB(10,  3,  4) 

DOUBl£  PRECISION  S (10,  3,  4)  ,P(10,  4)  ,SB(  10,  3,  4) 

DOIBIE  HIECISION  SBT(10, 3 ),Q(10,  3),R(10,  3),B(10,3,4) 

DOUBIE  PRECISION  PR  (4 ) ,Pl,  P2,  FZ  (4 ) ,FD  (10,  4)  ,PS  (4 ) 

INTECER  IPVr(10) 

DOUBIE  FRECI5I0N  AZESO,Aa>lE,  ATWO 

DOUBIE  PRECISION  RAD(10,  3)  ,RADINV(10,  3)  ,U(10,  3) 

DOUBIE  PRECISION  V(10,  3),W(10  ,3),Y(10,  3)  ,S(M,WGRK (10) 
Ca4MCN/INOU/KlN,  KDUT 
NAA=  3 

ATWO*  -3.  (£>0 
AZERO  = -4.  am 
AONE  * 6. 0D0 
Pl=  .05D0 
P2  = . 75D0 
KIN*  5 
KOin’=  6 
N-  3 
M*  3 
N2  * 6 
KOti  « 3 
NS*  10 
IPKT*  17 
lEND*  25 

icounr  * 0 

NSTEPS  « 25 
NA*  10 

I«A*  10 
NR*  10 
NB*  10 
NO*  10 
NG*10 

22  IFdCOUMT.rC.U)  READ  (KIN,  9500,  END*2 ) (PR  (I ) ,FZ  (I ) ,I*1,N) 

9500  FORMAT  (3D 25. 15) 

DO  11  JK*  1,N 
DO  11  JL  « 1,N 
Q(JL,  JK)  * 0.  £D0 
R(JK,X)  * 0.0D0 
11  A(JL, JK)  « 0.  d)0 
BEE  « -10.  eD0 
P(l,l)  * 1.(JD0-P1 
P(2,2)  ■ 1.0DO-  P2 
P(3,3)  - l.®0 
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P(l, 

2) 

e 

0. 

030 

P(l, 

3) 

X 

d. 

000 

P(2, 

1) 

» 

PI 

P(2, 

3) 

= 

0. 

000 

P(3, 

1) 

* 

0. 

030 

P(3, 

2) 

* 

P2 

A(l, 

1)' 

X {d.a)id 

A(2, 

2)‘ 

B 0.£D0 

A(3, 

3) 

= 

-AZE3R0 

A(l, 

2) 

s 

1. 

030 

A(2, 

3) 

s 

1. 

030 

A(3, 

1) 

s 

-ATWO 

A(3, 

2) 

= 

-PiOtE 

U(l, 

1) 

s 

).fcD0 

Q(2, 

2) 

s 

3. 

030 

Q(3, 

3) 

* 

3. 

030 

R(l, 

1) 

8 ] 

L.iD0 

R(2, 

2) 

s 

1. 

030 

R(3, 

3) 

s 

1. 

030 

Q(l, 

1, 

l)s  0a.£D0 

B(2, 

2, 

1) 

» 

0.030 

B{2, 

1, 

1) 

= 0 

1.  030 

B(l, 

2, 

1) 

> 0 

1.030 

B(l, 

3, 

1) 

• 0 

1.  030 

B(2, 

3, 

1) 

=0 

1.  030 

B(3, 

3, 

1) 

■ 

1.030 

B(3, 

1, 

1) 

= 

1.030 

B(3, 

2, 

1) 

* 

1.030 

B(l, 

1. 

2) 

= 

0.030 

B(2, 

2, 

2) 

S 

0.  030 

B(2, 

h 

2) 

*0 

1.030 

B(l, 

2, 

2) 

s 

0.  030 

B(l, 

3, 

2) 

= 

0.  030 

B(2, 

3, 

2) 

s 

0.  030 

B(3, 

*3 

’•'0 

2) 

s 

BEE 

B(3, 

1, 

2) 

s 

1.  030 

B(3, 

2, 

2) 

= 

1.030 

B(l, 

1, 

3) 

s 

0.030 

B{2, 

2, 

3) 

» 

0.  030 

B(2, 

1, 

3) 

s 

0.030 

B(l, 

2, 

3) 

a 

0.030 

B(l, 

3, 

3) 

3 

0.030 

B(2, 

3, 

3) 

3 

0.  030 

B(3, 

3, 

3) 

3 

0.  030 

B(3, 

1, 

3) 

3 

1.030 

B(3, 

2, 

3) 

1.030 

PR(1) 

M 

.0500 

PR  (2) 

m 

.7500 

P(l, 

1) 

m 

1. 

030  - PR  (1 ) 
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SWMAT  FtJRTRAN 
C lATblST  VERSION  2/17/78 

UOUBI£  HIECISION  E (4 ) ,ErEMP(4 ) ,SIM,  SIGMA,  SIGM 1,  ESINV,  ESIOl-A,  SINVMl 
DOUBI£  DECISION  CGND,  LUDOLF,  LUDINV,  EDLFMl,  EM(  10,  2) , (10)  ,DIN\iM  1 
DOUBLE  WBCISION  ARRAV:(100,  50)  ,Y0(10)  ,U0(10) 

DOUBLE  fRBCISION  \(10,  3)  ,C(10,  3) , Wl,  FR2,  X(10,  ?) 

Ed)  = 1.1D0 

EM(1,1)  = l.(I>0 
EM(2,  2)  = 1. 6D0 
E (2)  » 0.  (D0 
E'rEMP(2)  = E(2) 

E(3)  = 0.0)0 
ETEMP(3)  * E(3) 

ETEMP(l)  = Ed) 

E(4)  > 0.(D0 

DOUBLE  FRBCISION  QIORM ( 10,  2,  4) 

DOUBLE  FRBCISION  BSB(10,4,  3) 

DOUBLE  PRBCJSION  S (10,  3,  4)  ,01,  Pd0,  4)  ,S8(10,  3,  4) 

DOUBLE  FRBCISION  WR (4 ) ,WI  (4  ) ,HH  (4,  4) , XX  (4 , 4) , ACLd0,  3) 

DOUBLE  PRECISION  SBT (10,  3 ), 0(10,  3),Rd0,  3),B(10,2,4) 

DOUBLE  FRBCISION  PR  (4)  ,F2  (4)  ,FD  (10,  4)  ,15  (4) 

INTEOiR  IPVT  (10)  ,MCa4  (100)  ,hBTEPJ,  NGRIDH,  ICON  (100) 

DOUBIE  FRBCISION  RADd0,  5)  ,I?)DINV (10,  3)  ,SNBW(  10,  3,  4)  ,U (10,  3) 

DOUBLE  FRBCISION  V (10,  3)  ,W(10,  3)  ,W(10  , 3)  ,Y(10,  3)  ,SW,WORK (10) 

LOGICAL  NOISE 

COWCN/INOU/KIN,  lOUr 

K0=0 

lA  = 1 

READ(5, 11111)  NFUINT 
33333  READ(5, 11111,END=22222)  ITIME,  K 
11111  FORMAT  (21 4) 

DO  44444  IXYZ  * lA,  IIIME 
44444  MCCM(IXYZ)  » K0 
MCCN  (1TIME)=  K 
lA  « ITIME 
K0«  K 

00  lO  33333 

22222  DO  55555  IXYZ  * ITIME, NFOIMI 
55555  MCai(IXYZ)  = K0 

LUDOLF*  2. 718281828459045D0 

LUDINV*  l.a)0/LUDOLF 

DOLFMl  = LUDOLF  - 1.  ©0 

DIN\M1  * LUDINV  - 1.CD0 

NAA  >=  2 

NC  - 10 

KIN*  5 

KOUT*  6 

N*  2 

M*  2 

N2  « 4 
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ICON  « 4 
NH*  4 
NS«  10 
IPRr»  17 
1END=  50 
im  « 49 

iccxwr  « 0 

NA«  10 
Na«  10 

tKA*  10 
NR-  10 
NO-  10 
NG-10 

PZ(1)  - .1D0 
PR(1)-  .100 
DO  15  1-2, N 
PRU)-  PR(1) 

15  PZ(I)-  PZ(1) 

22  IP(ICCXMT.^£.0)  READ  (KIN,  9500,  EKD-2)  (PR  (I)  ,F0(I)  ,I-1,N) 
SIQIA-  1.0)0 
ESIOIA-  LUD0LF**Sia4A 
ESINV-  HJDINV**SIQ4A 
C(l,l)  - 1.0)0 
C(2,  2)  - 1.000 
C(l,2)  « 0.000 
C(2, 1)  - 0.0)0 

or  - 1.0)0 

NSTEK  - 50 
A(l,l)-  BSIGMA 
NAR-  100 
NAC  * 50 
A(2,  2)-  BSINV 
A(2,l)  « 0.0)0 
A(l,2)  -0.0)0 
0(1,1)-  14.0)0 
0(2,1)-  8.000 
0(1,2)  - 8.0)0 
0(2,2)  - 6.0)0 
R(l,l)-  1.0)0 
R(2,l)  - 0.0)0 
R(l,2)  - 0.0)0 
R(2,  2)  - 1.0)0 
B(l,l,  1)»  ESIOIA  -1.0)0 
B(2,l,  D-  ESINV-  1.0)0 
B(2,2,l)  - -B(2,l,l) 

B(l,2,l)-  B(l,l,l) 

B(l,l,2)  - 0.0)0 
B(2,2,  2)  - -DINVMl 


r 
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B(2, 1,  2)*  0.  £00 
B(l,2,  2)  » DOLFMl 
8(1,1,  3)  » DOIfWl 
3(1,2,  3)  « 0.  £00 
B(2,l,  3)  * 0IN\M1 
B(2,2,  3)  « 0.000 
PRl  * .100 
PR>  .100 
P(l,l)  * .8100 
P(2,2)  « .0900 
P(3,  2)  « 0.0900 
P(3,  3)  » .0900 
P(l,2)  * .8100 
P(3,l)  « .0900 
P(2, 1)  « .0900 
P(l,3)  » .8100 
P{2,3)  » 0.09OU 
P(4,l)  « .0100 
P(l,4)  » .8100 
P(4,2)  « .0100 
P(4,3)  » .0100 
P(4,4)  > .0100 
P(2,4)  » .0900 
P(3,4)  * .0900 
WRITE  (ROOT,  9903) 

CALL  MATIO  t4A,  KCM,  ICCN,  P,  3) 

C WRITE  (KOdr,  46) 

46  FORMAT  (/,4H  PI,/) 

47  FORMAT  (5325. 15) 

WRITE  (KOUT,  9600) 

CAU.  MATIO  C^A,  N,  N,  A,  3) 

WRITE  (KOl/T,  9700) 

CALL  MATIO  (NA,  N,  N,  (^,  3) 

WRITE  (KOUT,9800) 

CALL  MATIO  ftlR,  N,  N,  R,  3) 

DO  222  K«1,K:0N 

(W1  - K-1 

WRII'E(KOOr,9900)  KMl 

222  WRITE  (XDOT,  9500)  ((B(I,  J,K)  ,J«1,M)  ,I«1,N) 
44  CCRTINUE 
DO  14  IN-1,50 
LCON(IN)  « LCCN(l) 

14  CONTINUE 
667  FORMAT  (515) 

X0(1)  - .0200 

GN0RM(1,1, 1)«  -1.0633618400 
GNaRM(2, 1, 1)-  -7.  90151880-1 
GNORMd,  2, 1)»  -1.  887878890-02 
Q«RM(2,2,  D-  -5.835824960-02 


251 


SWMAT  FCRTRAN 

GNCMMd,  1,  2)=  -3.  69012096D-.J1 
GN0RM(2, 1,  2)=  -1.  14016534D0 
GN0RM(1,2,  2)»  1.04948339D-01 
a«RM(2,  2,  2)=  -1.363987670-41 
(2WW4(1,  1,  3)-  -1.  4256676700 
GNC»M(2, 1»  3)*  -2.874513080-01 
a«Rf4(l,2,  3)*  1.518842850-02 
a)ORM(2,2,  3)«  -7.270124380-02 
IR  » 2 
NPRft  = 1 
DO  57  IK  * 1,1CCN 
ll«l  * IK  - 1 
WRirEIKOirr,  9902)  IWIl 

57  W.9ITE(KOUr,950O)  ( (GNORM(IJ,  IL,  IK)  ,1L=1,N)  ,IJ=1,N) 

NGRIOH  > 5 
V(l,l)  « B(l,l,l) 

V(2,2)  » B(2,2,l) 

V(2,l)  > B(2,l,l) 

V(l,2)  * B(l,2,l) 

C CAIL  MMlL(NA,rA,M^,N,N,M,V,GNOBM,U) 

C CAIL  MAtX>C^,I«\,t^,N,N,U,A,^L) 

C CAIL  MSCAIE  «G,  N,  M,  -1 . I£>0,  QJC»M) 
lONE  = 1 

C CALL  MMl]Le^,N,N,  IONE,IR,N,C,  X0,Y0) 

65  FORMAT  (IX,  3)25. 15) 

C CALL  OR(BlM(NA,NO,NG,NAR,rAC,N,  IR,M,M:L,C,GNORM,X0,WORK, 

C 1Y,U,  IFVT, ARRAY,  or,  hBrEPB,NPRH,) 

C CAIL  READY2(NAA,^l\,^B,NQ,^R,NG,NS,^«A,N,M,fC0N,A,B,R,0,P, 

C 1 WR,WI,S,SD,U,V,W,X,  Y,Q40RM,FAD,R\0INV,BSB,waiK,IFVr,  lENO) 

DT  = 1.  0)0 
X0(1)  > .0200 

C CALL  MSCAIE  (NG,N,M, -1.0)0,  O^RM) 

X0(2)  » 0.0)0 

CALL  SWITCH  (NA,^B,^C,NG,NAR,^^C,N,  IR,  lAA,  K:GN,M,  A,B,  P, 

1 C,  GNOHM,  X0,  E,  BTfcMP,  EM,WCRK,  Y0,  U0,  V,W,  W,  IPVT,  ARRAY, DT,  ^BTEIB, 

2 NGRIOH, MCCN) 

9500  FORMAT  (2025. 15) 

2000  F0R4AT(/,3O25.  15) 

9600  FORMAT  (/,  31  A ) 

9700  FORMAT  (/,  31  Q ) 

9800  FORMAT!/, 31  R ) 

9900  FORMAT!/, 31  B ,15,/) 

9903  F0R4AT(/,31  P ) 

9902  FORMAT!/, 31  G ,15,/) 

2 STOP 
END 
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C lATEST  VERSION  2/17/78 

DOUBIE  PRECISION  E (4 ) ,ErEMP(4 ) ,SIM,  SIGMA, SIQ41,  ESINV,  ESIOIA,  SINVMl 
DOUBIE  PRECISION  CCND,  lEDOLF,  UJDINV,  DOLPMl,  EM(10,  2)  ,X0  (10)  ,niNVMl 
DOUBIE  PRECISION  ARRAY  (100,  50),  Y0  (10),  U0  (10) 

DOIBIE  PRECISION  A(10,  3)  ,C(  10,  3)  ,mi,  FR2,  X(10,  3) 

Ed)  = 1.0D0 
EM{1,1)  = l.®0 
EM(2,  2)  - 1.000 
E(2)  = 0.  0D0 
E'rEMP(2)  = E(2) 

E(3)  = 0.0D0 
E'TEMPO)  = E(3) 

ETQ4P(1)  = E (1) 

E(4)  = 0.000 

DOUBLE  PRECISION  GNORM(10, 2, 4) 

DOIBIE  PRECISION  BSB(10,  4,  3) 

DOIBIE  PRECISION  S (10,  3,  4)  ,01,  P(10,  4)  ,SB(10,  3,  4) 

DOUBLE  DECISION  WR  (4)  ,WI  (4)  ,HH  (4,  4)  ,XX  (4,  4)  ,ACL(10,  3) 

DOUBLE  DECISION  S8T(10,  3 ),Q(10,  3),R(10,  3),B(10,  2,4) 

OOIBLE  HIECISION  PR  (4)  ,PZ  (4 ) ,F0  (10,  4)  ,15  (4 ) 
lOTEGER  IFVr(10),MCCN(100),NPOINr,NGRIDH,lCON  (100) 

DOUBLE  reflCBION  RAD(10,  3)  ,RADINV (10,  3)  ,St®W(10,  3,  4)  ,U  (10,  3) 

DOUBLE  PRECISION  V(10,  3) , VW(10,  3)  ,W(10  ,3),Y(10,  3) , SIM, WORK  (10) 

LOGICAL  NOISE 

CCMMCN/INOUAIN,  lOUT 

K0>0 

lA  = 1 

READ  (5, 11111 )NPOINT 
33333  READ(5,  11111,END=22222)ITIME,K 
11111  FORMAT  (21 4) 

DO  44444  IXYZ  * lA, TTIME 
44444  MCCN  aXYZ)«K0 
MCCN  (ITIME)=K 
lA  * ITIME 
K0>K 

GO  TO  33333 

22222  DO  55555  IXYZ  * ITIME,  NPOIWT 
55555  MCCUaXYZ)  = K0 

LUDOLF»  2.  718281828459045D0 
ludinv>  i.a:>0/LUDoif’ 

DOLfMl  » LUDOLF  - 1.  0D0 

DINVMl  « LUDINV  - 1.000 

NAA  « 2 

NC  * 10 

KIN»  5 

KOUr*  6 

N*  1 

M»  1 

N2  « 2 
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ICCN  * 2 
NH«  4 
NSe  lU 
IPRT*  17 
IEND=  SU 
IPKl’  * 49 

iccxwr  - J 

NA-  10 
N3»  10 
(t1>NA 

10 

NR*  10 
NO-  10 
NG-10 

PZ(1)  » .1D0 
PR(1)*  .1D0 
DO  15  I^,N 
PR  (I)*  PR(1) 

15  PZ(I)-  PZ(1) 

22  IF(1COUW^.^C.0)  READ  (KIN,  9500,  END-2)  (PR  (I ) ,PZ  (I ) , 1*1  ,N) 
SIQIA*  l.eD0 
ESIOIA-  LUDOLF**SI(NA 
ESINV*  LUDINV**SI(NA 
C(l,l)  - l.tD0 
C(2,2)  - 1.ED0 
C(l,2)  * 0.(D0 
C(2,l)  * 0.0D0 
Dr  « 1.0D0 
A(l,l)«  1.4140D0 
NAR*  100 
NAC  - 50 
0(1,1)  - 3.030 
R(l,l)«  1.030 
R(2,l)  - 0.030 
B(l,l,l)-  2.030 

B(l,l,  2)-  .5D0 
P(l,l)  « .■»0 
P(2,  2)  * .7D0 
P(3,2)  « 0.  09D0 
P(3,  3)  - .0900 
P(l,2)  - .330 
P(3,l)  - .0900 
P(2, 1)  * . 330 
P(l,  3)  - .8100 
P(2,  3)  * 0.0900 
P(4,l)  - .01D0 
P(l,4)  « .8100 
P(4,2)  « .01D0 
P(4,3)  - .0100 
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F(4,4)  > .idlD0 
P(2,4)  = .M9D0 
P(3,4)  = .09D0 
WRITE  (KO(jr,99ia3) 

CALL  MATIO  (NA,  ICCN,  ICCN,  P,  3) 

C WRIIE  (KOUr,  46) 

46  FCmAT(/,41  PI,/) 

47  FORMAT  (3D  25.  15) 

WRITE  (XOUT,  9600 ) 

CAUL  MATIO  (NA,N,  N,A,  3) 

WRIIE  (KOl/T,  9700 ) 

CALL  MATIO  (NA,  N,  N,  Q,  3) 

WRITE  (KOUr,9800) 

CALL  MATIO  (NR,  N,  N,  R,  3) 

DO  222  K=1,ICCN 

KMl  = K-1 

WRITE  (KOUT,  9900)  KMl 

222  WRITE  (KOUT,  9500)  ( (B(I,  J,  K) , J=1  ,M)  , 1=1  ,N) 

44  CONTINUE 

QWRM(1,1,1)  = -1.  06  3 36  1 84D0 
GNORM(l,2,  1)  = -1.  88787889D-02 
GNOft4(2, 1, 1)  = 7.  90151884D-01 
GNORM(2,  2, 1)  = -5,83582460-02 
GLORM(l,l,  2)  = -3.  6901209&D-01 
GN0R(4(1,  2,  2)=1.  04948339D-01 
GN0RM(2,1,  2)  = -1.  14016354D0 
GN0RM(2  , 2,  2)  = -1.  36  308767D-01 
GN0HM(1,1,  3)=  -1.42566767D0 
GN0RM(2, 1, 3)=  -2. 87451308DHai 
GN0RM(2,  2,  3)  = -7.  27012438-02 
GN0RM(1,2,  3)  = 1.  51884285D-02 
GN0RM(1,1,  4)  = 0.a)0 
GNC«M(2,  2,  4)  = 0.0D0 
GN0RM(1,  2,  4)  = 0.eD0 
GN0RM(1,2,  4)  = 0.  0D0 
DO  14  IN=1,50 
LCCN  (IN)  = UCCH  (1) 

14  CCNIINUE 
667  FORMAT  (515) 

X0(1)  = .02D0 

IR  = 1 

NPRIL  = 1 

DO  57  IK  = 1,FCCN 

IW41  = IK  - 1 

WRITE  (ROOT,  9902)  IlMl 

57  WRITE  (KOUI,  9500)  { (GNORM  (IJ,  IL,  IK) , IL=1 , N) , 1J=1 , N) 
NGRIDH  = 5 
V(l,l)  = B(l,l,l) 

V(2,2)  = B(2,2,l) 
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V(2,l)  = 8(2, 1,1) 

Va,2)  » 8(1,  2,1) 

C CAU,  MI-ILL  (NA,  MV,  M\,N,N,M,V,aK)RM,U) 

C CALL  MAIX)(NA,NA,  MV,N,N,  U,A,a:L) 

C CAIL  MSCAIE  NG,N,M,-l.®k),(MORM) 

lONE  = 1 

C CAIL  MMIJL(NC,N,N,  IONE,IR,N,C,Xii),  Y0) 

66  FORMAT  (IX,  3D25. 15) 

C GAIL  DRCB1M(I'IA,NC,NG,NAR,NVC,N,  lR,M,ACL,C,a)ORM,X0,WORK, 

C 1Y,U,  IPVT,  ARRAY,  or,  NPOINT,NPRiL) 

C CAIL  R£ADY2(NAA,MV,^B,NQ,NR,  N3,NS,1«A,N,M,1CCN,A,B,R,Q,P, 

C 1 WR,W1,S,SB,U,V,W,X,Y,Q^0R-1,R^D,RADINV,BS6,WCRK,  IPVT,  lEND) 

OT  = 1.  eoB 

X0(1)  = .0200 

C CAIL  MSGAIE  (NG,N,M, -1.0)0,  GNORM) 

X0(2)  = 0.000 

CALL  SWITCH  (NA,  hB,  NC,  NG,  NAR,  NVC,  N,  IR,NVA,ICCN,M,  A, B,P, 

1 C,  (WORM,  Xa,  E,  ETEMP,  EM,WCRK,  Y0,  U0,  V,W,  VW,  IPVT, ARRAY,  DT,  NPDIOT, 

2 NGRIDH,MCCN) 

95J0  FORMAT  (2025. 15) 

2J30  FORMAT!/, 30  25.  15) 

9600  FORMAT  (/,  21  A ) 

9700  FORMAT  (/,3H  Q ) 

9300  FORMAT  (/, 31  R ) 

9900  FORMAT!/, 31  B ,15,/) 

9903  FORMAT!/, 31  P ) 

9902  FORMAT!/, 31  G ,15,/) 

2 Slop 

ENO 


I 


» 1 


( ! 
ii 
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